Developer Guide#

This guide is for developers extending mesofield — writing custom device adapters, subclassing Procedure, building frame processors, or contributing to the framework itself. If you’re just running acquisitions, see the User Guide instead.

Architecture in one diagram#

experiment.json
     │
     ▼
ExperimentConfig ─── HardwareManager ─── Devices (BaseDevice subclasses)
     │                     │                       │
     │                     │                       ├── DataProducer.signals.data ──┐
     │                     │                       │                               ▼
     │                     └── DataManager ◀───── Queues ─────── DataSaver ───--- disk
     │
     ▼
Procedure (orchestrates lifecycle, emits signals, owns the run)
     │
     ▼
MainWindow (Qt GUI; binds widgets to procedure.events)

The four backbone classes are:

Class	Module	Owns
`Procedure`	`mesofield.base`	Run lifecycle (initialize → arm → start → finish), hooks, manifest
`ExperimentConfig`	`mesofield.config`	Parameter registry + JSON I/O + BIDS paths
`HardwareManager`	`mesofield.hardware`	YAML-driven device factory + lifecycle
`DataManager`	`mesofield.data`	Per-run data queues, notes, timestamps, manifest writes

Procedure lifecycle#

initialize_hardware       — bring devices up (one-time)
data.setup + register_devices + start_queue_logger
gate stimulus devices by task
prerun                    — subclass hook (default: no-op)
hardware.arm_all          — per-run prep on every device
await_trigger             — subclass hook; hold for an external trigger
connect primary.signals.finished -> _cleanup_procedure
hardware.start_all        — procedure_started emits just before
on_started                — subclass hook
arm the wall-clock duration cap
on_finished              — subclass hook (primary finished, or cap fires)
save_data + manifest write + cleanup

Hooks prerun, await_trigger, on_started, on_finished are no-ops on Procedure itself. Override them in your subclass’s procedure.py (mesofield init scaffolds one). manifest_extra is the hook for adding your own keys to the written AcquisitionManifest.

Cleanup is latched: whichever fires first — the primary device’s finished signal or the duration timer — tears down exactly once. run_until_finished(timeout=None) blocks until that happens, which is what the scaffolded python procedure.py entry point calls.

from mesofield.base import Procedure


class MyProcedure(Procedure):
    def prerun(self):
        self.logger.info(f"Subject {self.config.subject}, "
                         f"duration {self.config.duration}s")

    def on_started(self):
        # called after every device has started
        pass

    def on_finished(self):
        # called after the primary camera signals finished
        self.logger.info("Run complete; manifest will be written next")

load_procedure is the discovery hook called by the CLI. Launching a procedure.py instantiates your subclass, which loads its self-contained experiment.json (parameters plus an embedded hardware rig block) via the class-level experiment path.

Procedure signals (`procedure.events`)#

Procedure.events is a ProcedureSignals QObject exposing five pyqtSignals:

Signal	Payload	Fires when
`procedure_started`	—	After all devices have started
`hardware_initialized`	`bool` (success)	After `initialize_hardware`
`procedure_finished`	—	After cleanup completes successfully
`procedure_error`	`str` (message)	On any uncaught run-time error
`data_saved`	—	After `save_data` completes

Connect from a Qt widget or from another device:

procedure.events.procedure_started.connect(self.lock_form)
procedure.events.procedure_finished.connect(self.unlock_form)
procedure.events.procedure_error.connect(self.show_error_dialog)

Custom hardware devices#

A hardware device is any class that satisfies the HardwareDevice protocol. The easiest path is to subclass one of the base classes:

Base	Use when
`BaseDevice`	Generic device with no streaming data
`BaseDataProducer`	Streaming source (timeseries / counts) with a buffer
`BaseSerialDevice`	Streaming source whose transport is a serial port
`BaseCamera`	Anything that produces frames + writes a writer file

Minimal example — a serial sensor#

from mesofield import DeviceRegistry
from mesofield.devices.base import BaseSerialDevice


@DeviceRegistry.register("thermal")
class ThermalSensor(BaseSerialDevice):
    """One-byte-per-sample thermal probe over serial."""

    file_type = "csv"
    bids_type = "beh"
    data_type = "thermal"

    def parse_line(self, line: bytes):
        """Parse one serial frame.

        Returns:
            ``(payload, timestamp_or_None)`` — the payload is whatever
            you want fanned out on ``self.signals.data``; pass ``None``
            for the timestamp to let the framework stamp it.
        """
        return float(line), None

Then in hardware.yaml:

thermal:
  type: thermal
  port: /dev/ttyUSB1
  baudrate: 115200
  output:
    suffix: thermal
    file_type: csv
    bids_type: beh

@DeviceRegistry.register("thermal") is what binds the YAML type: string to the class. The decorator also stamps registry_key onto the class so any instance can report its YAML type for hardware export.

Camera-shaped devices#

For anything that produces frames, subclass BaseCamera — it defaults to OME-TIFF output via CustomWriter, exposes a snap() / start_live() / stop_live() contract for the GUI preview, and plumbs frame metadata into the manifest. The MMCamera and OpenCVCamera classes are the two concrete implementations to read for reference.

Threading models#

Devices can use any concurrency model that respects the lifecycle:

# Qt-thread device (camera, GUI-driven serial)
from PyQt6.QtCore import QThread
from mesofield.protocols import HardwareDevice

class QtDevice(QThread):
    device_type = "qt_device"
    device_id   = "qdev"
    def run(self): ...     # Qt thread loop


# Python threading device
from mesofield.protocols import ThreadedHardwareDevice

class ThreadedDevice(ThreadedHardwareDevice):
    device_type = "thread_device"
    device_id   = "tdev"
    def _run(self): ...    # standard daemon thread


# asyncio device
from mesofield.protocols import AsyncioHardwareDevice

class AsyncDevice(AsyncioHardwareDevice):
    device_type = "async_device"
    device_id   = "adev"
    async def _run(self): ...

Pick whichever fits your hardware best — the framework only cares about the protocol, not the concurrency model.

Frame processors#

For per-frame compute (mean intensity, ROI tracking, anything that turns an ndarray into a scalar), use the @processor decorator on a Procedure method:

from mesofield.base import Procedure, processor


class MyProcedure(Procedure):
    @processor(camera="meso", plot=True, label="Frame mean", y_range=(0, 65535))
    def frame_mean(self, img, idx, ts):
        return float(img.mean())

The framework wraps the function in a FrameProcessor, attaches it to the camera whose device_id matches "meso", registers it with DataManager, and (when plot=True) adds a live SerialWidget to the GUI.

Recognised plot_kwargs: label, value_label, value_units, y_range, value_scale, max_points.

Scaffolding a new experiment#

The CLI scaffold drops a fill-out template:

mesofield init my-experiment --rig my-rig
cd my-experiment

You get:

my-experiment/
    README.md
    experiment.json      # subjects, duration, DisplayKeys
    hardware.yaml        # copied from the selected rig
    procedure.py         # your Procedure subclass
    devices/
        __init__.py
        thermal_example.py  # annotated custom-device template

--rig selects from mesofield rig list. Use --hardware path/to/file to use an explicit YAML; omit both to enter an interactive picker.

Rig store#

A hardware.yaml is rig-specific (COM ports, camera ids, MM .cfg paths). Each machine keeps a small store of named canonical configs in the OS config directory:

mesofield rig new my-rig             # writes a fill-out template
mesofield rig list                   # show registered rigs (and their devices)
mesofield rig add my-rig file.yaml   # adopt an existing yaml
mesofield rig show my-rig            # print a registered rig
mesofield rig remove my-rig

The store lives at the platform-default config location (resolved by platformdirs); mesofield rig list prints the path it is reading.

Downstream extension points#

Acquisition is only the first layer in the package. Two more are worth knowing when you add a device:

Parsers — mesofield.datakit.sources.register.TimeseriesSource. Subclass it, set tag and patterns, implement build_timeseries, and bind it to your producer as MyDevice.Parser = _MyParser so manifest-driven dispatch finds it. Without a parser your device writes files that ingest can’t read back.
Processors — mesofield.processing.ProcessorRunner, for any file-to-file stage between acquisition and ingest (DLC, mesomap, lab pipelines). Set tool_name / tool_version and implement run(inputs, **params) -> list[Path]; calling the instance returns (outputs, manifest) and writes a <tool_name>.process.json sidecar recording input hashes, parameters, and upstream provenance.

Logging#

Use the project logger inside your code:

from mesofield.utils._logger import get_logger

logger = get_logger(__name__)
logger.info("...")

Every device and processor should use a __name__-scoped logger so the file traces are easy to grep.

Where to look for examples#

mesofield/devices/mocks.py — mock serial encoder + mock camera. Read these first; they’re the simplest concrete implementations of every base class.
mesofield/devices/cameras.py — MMCamera (Micro-Manager backend) and OpenCVCamera (capture thread + MP4 writer). The two shapes most custom cameras start from.
mesofield/scaffold/experiment.py — what the mesofield init CLI emits. Reading the templates is a shortcut to understanding the expected file shape.
mesofield/processors/frame_mean.py — three-line frame processor. The minimum viable processor.

Retrofitting legacy sessions#

Sessions acquired before manifests landed can be retrofitted:

mesofield process retrofit-manifest /path/to/experiment

This walks the BIDS tree, reads timestamps.csv / configuration.csv, and synthesises an AcquisitionManifest per session. Calibration constants aren’t recoverable (they weren’t written), but everything else round-trips and downstream tools become happy again.