Commit Graph

141 Commits

Author SHA1 Message Date
102a11eed7 Add beamtime functionality to backend.
Introduce new endpoint and model for managing beamtimes, including shifts and user-specific access. Updated test scripts and data to reflect beamtime integration, along with minor fixes for job status enumeration and example notebook refinement.
2025-05-05 16:05:37 +02:00
b13a3e23f4 Add job cancellation handling and periodic cleanup logic
Introduce new statuses, "to_cancel" and "cancelled", to improve job state tracking. Implement logic to nullify `slurm_id` for cancelled jobs and a background thread to clean up cancelled jobs older than 2 hours. Ensure periodic cleanup runs hourly to maintain database hygiene.
2025-05-02 10:48:54 +02:00
a1b857b78a Add job cancellation handling and periodic cleanup logic
Introduce new statuses, "to_cancel" and "cancelled", to improve job state tracking. Implement logic to nullify `slurm_id` for cancelled jobs and a background thread to clean up cancelled jobs older than 2 hours. Ensure periodic cleanup runs hourly to maintain database hygiene.
2025-05-01 15:17:42 +02:00
9e875c5a04 Update sample handling and experiment linkage logic
Added `type` to experiment runs in `sample.py` and improved filtering in `processing.py` to match experiments by both `sample_id` and `run_id`. Removed extensive unnecessary code in `testfunctions.ipynb` for clarity and maintenance.
2025-04-30 16:41:05 +02:00
b3847a0bf0 Fix job type assignment and clean up testfunctions file.
Updated job type to reference `experiment.type` in `processing.py` for accurate data handling. Cleaned up and streamlined `testfunctions.ipynb` by removing outdated and redundant code, improving readability and usability.
2025-04-30 13:09:55 +02:00
38a5c85b37 Fix job type assignment and clean up testfunctions file.
Updated job type to reference `experiment.type` in `processing.py` for accurate data handling. Cleaned up and streamlined `testfunctions.ipynb` by removing outdated and redundant code, improving readability and usability.
2025-04-30 10:56:45 +02:00
58dcaf892f Add 'type' field to ExperimentParametersModel
This commit introduces a new 'type' field in the ExperimentParametersModel schema and updates the associated code in `sample.py` to include this field during object creation. Additionally, unnecessary lines and redundant code in `testfunctions.ipynb` have been removed for better readability and maintainability.
2025-04-29 23:17:53 +02:00
57de665c7b Add dataset, slurm_id, and FAILED status to models
Enhanced the models with new fields: a dataset field for Experiment Parameters and a slurm_id for Jobs. Introduced a FAILED status for the JobStatus enum. Updated functionality to handle datasets and trigger job creation based on dataset status.
2025-04-29 22:52:36 +02:00
9af2e84f9e Refactor job model and endpoints for improved structure
Updated the job model to include `sample_id` and `run_id` fields, replacing `experiment_parameters_id`. Adjusted relationships and modified routers to reflect these changes. Added an endpoint for updating job status and restructured job streaming logic to include detailed experiment and sample data.
2025-04-29 14:43:39 +02:00
3eb4050d82 Refactor job model and optimize job streaming.
Updated the `JobModel` with foreign key relationships and string-based status to enhance database consistency, and improved job event streaming by using `jsonable_encoder` for better serialization. Also, streamlined dependencies by adding `urllib3` to handle HTTP requests.
2025-04-29 09:47:57 +02:00
866139baea Added a test function for SSE 2025-04-11 13:59:46 +02:00
2e6d06018c Update server config, SSL handling, and port mapping logic
Refactored `run_server` to accept explicit config and SSL paths. Added dynamic environment-based config loading and stricter SSL path checks for production. Updated `docker-compose.yml` to use environment variable for port mapping and adjusted `config_prod.json` to reflect correct port usage.
2025-04-11 13:20:34 +02:00
fda9142155 Add job processing system with streaming endpoint
Introduced a `processing` router to handle job streaming using server-sent events. Added `Jobs` and `JobStatus` models for managing job-related data, along with database creation logic. Updated the `sample` router to create new job entries during experiment creation.
2025-04-10 11:53:36 +02:00
bb6cca4f23 Refactor Docker setup and migrate to PostgreSQL
Streamlined Dockerfiles with clearer ENV variables and build args. Switched backend database from MySQL to PostgreSQL, updated configurations accordingly, and added robust Docker Compose services for better orchestration, including health checks and persistent storage.
2025-04-09 15:09:22 +02:00
36f9978c79 Update dependencies and improve Python path handling
Updated several frontend dependencies including MUI packages and added new ones like `@mui/x-charts`. Adjusted the Python path setup in the CI configuration to correctly point to the `aaredb` backend, ensuring accurate module resolution.
2025-03-19 11:51:49 +01:00
536cfcd34b Update dependencies and improve Python path handling
Updated several frontend dependencies including MUI packages and added new ones like `@mui/x-charts`. Adjusted the Python path setup in the CI configuration to correctly point to the `aaredb` backend, ensuring accurate module resolution.
2025-03-19 09:14:12 +01:00
68f87f0d8d Displaying Processing Results in the frontend 2025-03-17 16:45:50 +01:00
5a0047b6d5 Refactor AareDB backend and update schemas and paths.
Revised backend schema definitions, removing unnecessary attributes and adding new configurations. Updated file path references to align with the aaredb structure. Cleaned up redundant notebook content and commented out unused database regeneration logic in the backend.

Added posting a result to the database
2025-03-17 11:51:07 +01:00
fbc32474ff Refine event types and update related models and logic
Standardized event types with stricter validation using `Literal`. Adjusted related data and logic to align with new types, including changes to PGROUP assignments, event timeline increments, and schema updates. Cleaned up unused code and clarified database initialization behavior.
2025-03-14 13:11:05 +01:00
2e1d87c31b Update dependencies and migrate to Node 18 minimum support
Upgraded multiple package versions, including `@esbuild` and dependencies like `@mui/x-data-grid-premium`. Adjusted `node` engine requirement to `>=18` for compatibility. This ensures modernization and alignment with current toolchain standards.
2025-03-04 10:42:24 +01:00
548a86678b Update SampleTracker to support dynamic activePgroup
Enhanced the `SampleTracker` component to accept and utilize an `activePgroup` prop, allowing dynamic filtering of data based on the current project group. Adjusted related polling and data fetching logic to respond to changes in `activePgroup`. Removed excessive code from the test notebook for cleanup.
2025-02-27 11:20:04 +01:00
2f5cb3032a Add development-only user simulation using local file
This change introduces a fallback mechanism for development environments, allowing authentication to simulate a user from a local file named "user" when not found in the mock database. The file must include a username on the first line and a space-delimited list of pgroups on the second line. This enhancement helps streamline development workflows while maintaining error handling for missing or malformed files.
2025-02-27 10:45:16 +01:00
7a9cc24e05 Add development-only user simulation using local file
This change introduces a fallback mechanism for development environments, allowing authentication to simulate a user from a local file named "user" when not found in the mock database. The file must include a username on the first line and a space-delimited list of pgroups on the second line. This enhancement helps streamline development workflows while maintaining error handling for missing or malformed files.
2025-02-27 10:39:45 +01:00
b04c7b8c95 Add Image models and clean up test code structure
Introduced `ImageCreate` and `Image` models to handle image-related data in the backend. Improved the organization and readability of the testing notebook by consolidating and formatting code into distinct sections with markdown cells.
2025-02-26 15:11:20 +01:00
1606e80f81 Add Image models and clean up test code structure
Introduced `ImageCreate` and `Image` models to handle image-related data in the backend. Improved the organization and readability of the testing notebook by consolidating and formatting code into distinct sections with markdown cells.
2025-02-26 13:33:23 +01:00
f588bc0cda Add endpoint for creating local contacts with access control
Introduced a new `local_contact_router` to handle creation of local contacts. The endpoint enforces role-based access control and ensures no duplication of email addresses. Updated the router exports for consistency and cleaned up a large test file to improve readability.
2025-02-26 09:58:19 +01:00
43d67b1044 Enhance Dewar handling and display in logistics system
Added new fields and enriched data representations in DewarStatusTab, backend schemas, and APIs to improve dewar tracking and management. Introduced new API endpoint `/dewar/table` for simplified data rendering. Applied logging and validations for missing relationships.
2025-02-05 21:43:17 +01:00
25673ae05c added tabs to logistics frontend 2025-02-05 11:55:46 +01:00
9e5ae2b43c Refactor puck event handling and add tell filtering
Updated puck event queries to improve robustness and ensure distinct results. Introduced filtering by `tell` in specific API endpoints and added validation for `tell` values. Incremented project version to 0.1.0a21 to reflect API changes.
2025-02-04 17:07:25 +01:00
780ba1959f Add SetTellPositionRequest schema and minor cleanup.
Added a new `SetTellPositionRequest` schema in `schemas.py` to support bulk updates of TELL positions. Commented out redundant metadata operations in `main.py` and cleaned up unused content in the test notebook for better readability.
2025-02-04 14:43:59 +01:00
70962ba67a Refactor dewar query to use latest event subquery.
Replaces the previous query logic with a subquery to find the latest logistics event for each dewar. Ensures that only dewars with the most recent "beamline" event are retrieved, improving query accuracy and clarity.
2025-02-03 11:46:27 +01:00
b63f696543 Fix timestamp method and loop variable order issues
Updated `datetime.utcnow()` to `datetime.now()` to align with local timezone handling. Corrected the order of unpacked variables in the loop to match the structure of `pucks_with_latest_events`.
2025-01-31 14:57:06 +01:00
7ce8c50179 Fix timestamp method and loop variable order issues
Updated `datetime.utcnow()` to `datetime.now()` to align with local timezone handling. Corrected the order of unpacked variables in the loop to match the structure of `pucks_with_latest_events`.
2025-01-31 12:58:15 +01:00
3757950b51 Refactor Dewar service methods and improve field handling
Updated Dewar API methods to use protected endpoints for enhanced security and consistency. Added `pgroups` handling in various frontend components and modified the LogisticsView contact field for clarity. Simplified backend router imports for better readability.
2025-01-30 15:25:04 +01:00
c2215860bf Refactor Dewar service methods and improve field handling
Updated Dewar API methods to use protected endpoints for enhanced security and consistency. Added `pgroups` handling in various frontend components and modified the LogisticsView contact field for clarity. Simplified backend router imports for better readability.
2025-01-30 13:39:49 +01:00
44582cf38e Add pgroup handling in dewars and enhance ShipmentDetails UI
Introduced a new `pgroups` attribute for dewars in the backend with schema and model updates. Modified the frontend to display `pgroups` as chips, integrate new visual icons for pucks and crystals, and enhance the UI/UX in `ShipmentDetails` and `DewarStepper` components. Added reusable SVG components for better modularity and design consistency.
2025-01-23 13:57:25 +01:00
173e192fc4 Integrate pgroups for shipment data security
Added `pgroups` to secure and associate data with specific permission groups. Updated backend routers, database models, and API endpoints to handle authorization based on `pgroups`. Adjusted frontend components and hooks to support `pgroups` in data fetching and management workflows.
2025-01-22 22:53:37 +01:00
382b1eaba8 Refactor contact handling across backend and frontend
Replaced usage of "ContactPerson" with "Contact" for consistency across the codebase. Updated related component props, state variables, API calls, and database queries to align with the new model. Also enhanced backend functionality with stricter validations and added support for handling active pgroups in contact management.
2025-01-22 16:31:08 +01:00
6cde57f783 **Commit Message:**
Enhance app with active pgroup handling and token updates

Added active pgroup state management across the app for user-specific settings. Improved token handling with decoding, saving user data, and setting OpenAPI authorization. Updated components, API calls, and forms to support dynamic pgroup selection and user-specific features.
2025-01-22 13:55:26 +01:00
4630bcfac5 Add mount_count and unmount_count tracking for samples
Introduced `mount_count` and `unmount_count` fields to track mounting events for samples. Updated models, schemas, and front-end components to support dynamic calculation and display of these counts. Enhanced backend queries and API responses to include the new data.
2025-01-20 13:04:20 +01:00
3b315f2997 Add mount_count and unmount_count tracking for samples
Introduced `mount_count` and `unmount_count` fields to track mounting events for samples. Updated models, schemas, and front-end components to support dynamic calculation and display of these counts. Enhanced backend queries and API responses to include the new data.
2025-01-20 11:02:56 +01:00
3d804c1635 Update logistics data display and dewar event handling
Renamed "Retrieved Timestamp" to "Last Event" for clarity in the UI. Improved backend logic to handle dewar events, including associating refill events with specific slots and retrieving the last slot ID for new events.
2025-01-17 15:59:49 +01:00
6825421f20 Update package-lock.json with latest dependency versions
Upgraded multiple dependencies across Babel, Emotion, ESLint, and DevExpress packages in `package-lock.json` to their latest versions. These updates ensure compatibility, fix minor issues, and improve overall performance and security.
2025-01-17 15:35:29 +01:00
9739b8cfe9 Make shipment fields optional and refactor test scripts.
Updated the `number_of_pucks` and `number_of_samples` fields in the `schemas.py` to be optional for greater flexibility. Simplified the test Jupyter Notebook by restructuring imports and consolidating function calls for better readability and maintainability.
2025-01-17 09:36:16 +01:00
481068603b Enhance deletion processes with event-check validations.
Added validation to prevent deletion of shipments, dewars, pucks, or samples if they have associated events. Updated frontend components to handle and display error messages based on API responses for improved user feedback.
2025-01-15 15:30:54 +01:00
e63af3e66d ```
Add duplicate detection for spreadsheet data processing

Implemented logic to detect and handle duplicate 'positioninpuck' entries within the same puck during spreadsheet processing. Updated backend to validate duplicates and provide detailed error messages. Enhanced frontend to visually highlight duplicate errors and allow better user feedback during cell editing.
```
2025-01-15 14:23:51 +01:00
7861082a02 Set default values for empty "priority" column in spreadsheets.
Added logic to assign a default value of 1 to empty "priority" fields in the spreadsheet service. Adjusted the router to correctly track columns explicitly marked as defaulted.
2025-01-14 22:18:14 +01:00
c0951292d0 Refactor spreadsheet handling to track corrections and defaults
Improved the backend's value cleaning to differentiate between corrections and defaults, logging metadata for clearer traceability. Updated frontend to display corrected/defaulted fields with visual cues and tooltips for better user feedback. Enhanced data models and response structures to support this richer metadata.
2025-01-14 21:46:11 +01:00
f6c19cc4da Refactor spreadsheet processing to improve validation logic
Enhanced value cleaning and validation for spreadsheet data with dynamic handling of columns and corrections. Improved feedback for users with detailed error messages and visual indicators for corrected or defaulted values. Simplified backend and frontend logic for better maintainability and usability.
2025-01-13 21:55:15 +01:00
19ef20e6b0 Fix type and handle missing dewar_id in sample router.
Updated puck_id type from str to int for better type consistency. Added a fallback value of -1 for pucks with missing dewar_id to prevent potential errors when querying pucks.
2025-01-10 13:39:10 +01:00