NSSP Bacteria Analysis Application

Description

  1. MS SQL Server Database: Collaborate with DOIT to establish the database structure in MS SQL Server or secure the permissions needed to manage it independently.

  2. Database Connectivity: Test and confirm connectivity with SQL Server.

  3. Migrate and Reformat Supporting Datasets: Database structure for relating the core bacti and rainfall data.

  4. Admin Backend for Updating the Database:

    1. Grant application-level user permissions and roles for various access levels to the SQL Server database

    2. Have controls that import bacteria and rainfall data into the SQL Server database

  5. Develop Web UI Screens: Used for selecting subset of data records, base analysis criteria, and reports to be generated.

  6. Statistical Analysis: Implement packages and coding logic to replicate the current MS Access application's procedures.

  7. Report generation: Tool for exporting analysis and summary reports that can be directly inserted into FDA shellfish reports.  Upwards of 15 unique types of reports will be created.

  8. Spatial Analysis Prep:  Design logic to generate temporary tables which are created each time a unique combination of criteria are ran using the Web UI screens.  These temporary tables are for the ESRI customized ArcGIS project to consume, enabling the spatial representation of the statistical data.

  9. Customized ESRI NSSP Analysis Tool: Although considered Phase II, it's included here because the project isn't complete until it's updated and fully functional by spatially representing the data results.

Problem Statement

DOIT has requested that Microsoft Access be phased out as a primary solution for networked data storage and as a user interface tool for data entry. Consequently, any future development for systems using Access must transition to a different platform. We need to redesign our customized MS Access system, which currently manages all statistical analysis and reporting for the NSSP program. Our goal is to develop this new system internally.

The program plans to redevelop BMWM’s BactiAnalysis system using the Python Django framework, and has already received verbal approval from Kurt Jaegers at DOIT. Python is a popular language for data analysis, offering numerous packages for various functionalities, ensuring reliability and longevity. Django, a Python-based web framework, provides built-in templates, modules, and security features, ideal for connecting to SQL Server tables and offering an admin backend for data modification. Many WRM staff are familiar with Python, simplifying support and hiring of staff if needed in the future.

Project Justification

• Shellfish Growing Water Classification rules through regulations (N.J.A.C. 7:12).
• New Jersey participates in the National Shellfish Sanitation Program (NSSP),
recognized by the Interstate Shellfish Sanitation Conference (ISSC; NJ is a
member of the conference) and the U.S. Food and Drug Administration (FDA).
• The NSSP, among other things, develops and updates regulatory guidelines for
the sanitary control of shellfish produced and sold for human consumption.

Estimated Transactions

Staff use this analysis program throughout the year to conduct various combinations of statistical analysis on data. Hard to estimate how often it is used but its a lot.

Created

8 July 2025, 08:55

Target Rollout Date

1 January 2027

Target Rollout Date Reason

The program aims to establish a solution soon, as they're aware of issues in other DEP programs using custom MS Access applications. Application to be written and supported by WRM staff.

Updated

26 February 2026, 16:17

Attachments

Activity

Knute Jensen 26 February 2026, 21:17

Meetings 2/11 and 2/18 (recorded) in DOIT led to a path forward with provisioning a VM under some new terms and guidance with a middle-term objective to use the questions that arise from standing this project up to formulate more general governance and guidance for in-house, in-program development efforts. Here is a meeting summary:

Generated by AI. Be sure to check for accuracy.

Meeting notes:

  • Web Application Environment Setup and Data Migration: Neal, Knute, Kurt, and Jim discussed the setup of a Django web application environment, including the migration of backend data to SQL Server and the provisioning of VMs, with Neal clarifying the current status and requirements for production and development environments.

    • Proof of Concept Review: Neal explained that the initial work involved setting up a basic Apache web server and sharing a folder for the Django web app, which was used to test connectivity and data display without advanced features or process integration.

    • Backend Migration Status: Neal clarified that some backend data had already been migrated to SQL Server, which the Django application is currently accessing, but was uncertain about the extent of migration and suggested Mike could provide more details.

    • VM Provisioning Requirements: Neal and Kurt discussed the need to provision VMs for the application, with Neal noting that two environments (production and development) may be required, and Kurt suggesting that both could run on the same physical box as separate web servers.

    • Server and Software Specifications: Kurt and Neal clarified that Apache, not Tomcat, is being used for the Python-based web app, and that the server specifications provided were based on existing standards but could be simplified for this use case.

  • Roles, Responsibilities, and Governance for Application Development: Knute, Kurt, and Neal addressed the division of responsibilities between infrastructure and program teams, outlining which aspects of server and application management fall to each group and discussing the need for clear governance and documentation for future projects.

    • Responsibility Breakdown: Kurt reviewed a list of ten items related to Python server environments, specifying which responsibilities (such as OS-level hardening and vulnerability scanning) fall to the infrastructure team and which (such as code updates and secure coding practices) fall to the program team.

    • Governance Documentation: Knute and Kurt discussed the importance of providing clear documentation and guidelines to teams receiving VM space, including a checklist of requirements and security practices to ensure proper governance.

    • Future Collaboration: Jim suggested that Neal and Kurt collaborate on developing an approved list of tools, applications, and guardrails for development projects, with Neal agreeing to work with Kurt and keep Michelle informed.

  • Source Code Management and GitHub Licensing: Kurt recommended mandating GitHub licenses for all development projects, ensuring code is stored in the DEP enterprise GitHub repository for visibility, auditability, and continuity, with agreement from Knute and Jim and discussion of cost and access controls.

    • GitHub License Mandate: Kurt proposed that all developers working on these projects be required to purchase GitHub licenses and store their code in the DEP enterprise GitHub repository, allowing for oversight and audit capabilities.

    • Access and Isolation: Kurt explained that code repositories would be isolated by program area, preventing cross-access between teams while allowing sharing within each organization.

    • Cost and Billing: Kurt and Jim discussed the cost of GitHub licenses, noting they are approximately $250 per year and less expensive than Adobe licenses, with costs billed back to the respective teams.

  • Python Environment Management and Use of Docker: Kurt and Neal discussed the management of Python environments, recommending that program teams maintain their own virtual environments and consider using Docker for environment separation, with Neal and Knute exploring the implications for VM provisioning and admin access.

    • Virtual Environments: Kurt explained the benefits of Python virtual environments for running multiple systems on the same server, allowing teams to test and switch between different Python versions without conflicts.

    • Docker Recommendation: Kurt suggested using Docker to manage separate environments within a single VM, reducing the need for multiple VMs and simplifying administration by granting teams admin access to containers rather than servers.

    • Admin Access Considerations: Neal asked about the need for admin access, and Kurt clarified that admin rights should be limited to Docker containers to avoid broader server access, aligning with OIT preferences.

  • Technology Standards and Project Tracking: Knute, Kurt, and Jim discussed the need to establish and track approved languages, technologies, and development platforms, distinguishing between programmatic and citizen development, and planning to monitor project portfolios and technology usage.

    • Approved Technologies List: Kurt indicated that a list of advised languages and technologies would be developed, requiring justification for deviations from established standards when new projects are proposed.

    • Project Portfolio Tracking: Knute described efforts to track IT projects by development type and responsible party, aiming to distinguish between internal, vendor, and hybrid development efforts for better portfolio management.

  • Low-Code Tools and Developer Qualifications: Knute, Kurt, and Jim reflected on the increasing use of low-code tools and AI-assisted coding, raising questions about developer qualifications, training, and auditing as teams adopt new technologies and practices.

    • Low-Code Adoption: Knute shared examples of AI-assisted code generation and discussed the efficiency gains, while Kurt and Jim cautioned about the risks of unqualified users relying on generated code without proper verification.

    • Developer Qualification and Auditing: Knute raised concerns about defining qualifications, required training, and audit processes for teams engaging in development, noting that these governance challenges will need to be addressed as the organization moves forward.

Follow-up tasks:

  • VM Provisioning for Django Project: Provision a VM (or VMs) for the Django project, considering the recommendation to use Docker for environment separation instead of multiple VMs, and communicate this approach to the project team. (Neal, Kurt)

  • Approved Tools and Technologies List: Collaborate to create and finalize a list of approved tools, applications, and technologies for programmatic development projects, including guidance and guardrails for project teams. (Neal, Kurt)

  • Project Tracking and Portfolio Distinction: Develop a method to track IT projects, distinguishing between programmatic development, citizen development, and vendor involvement, to clarify the portfolio and responsibilities. (Knute)

  • Governance Documentation for Python Projects: Draft and distribute governance documentation (including the list of 10 key items and any additional requirements) to all teams receiving VM space for Python development, ensuring they are aware of responsibilities and standards. (Knute, Kurt)

  • GitHub License Management and Policy: Establish and communicate a policy mandating the purchase and use of GitHub licenses for all development teams, ensuring code is stored under the DEP enterprise GitHub for visibility and audit purposes. (Kurt)

Mike.Kusmiesz 3 February 2026, 16:36

Over the past several months, Neal Krause has collaborated with Marine Water Monitoring staff to establish a Python/Django environment. This involved configuring and testing the necessary infrastructure and software to support application development. Testing was successful but conducted only on a development server for baseline testing.

We are now ready to transition from the development test server to actual Development and Production Virtual Machines (VMs). I am officially making the request for assistance from DOIT to initiate the setup of these VMs with OIT. Neal Krause is closely involved with the project and can provide further details if needed.

If there is a need for me to write up any justifications or supplemental documentation, please let me know and I will get that too you.

These are the specifications DOIT tested, and we anticipate they will suffice for the bureau’s future development tools.

Windows Server 2022

16gb Dynamic RAM

120gb C: drive (or whatever the standard is)

4 vCPUs

Knute Jensen 25 September 2025, 18:06

Today I reinforced with Mike that DOIT is supportive of the program going forward with development as described.

Mike.Kusmiesz 9 September 2025, 13:25

WMSPC staff have already met with Kurt Jaegers to discuss the proposed platform to develop this in. Django / Python. It is through this Project IT Sheet we are formalizing the need for this project and would like approval from DOIT to begin it ASAP.