Skip to content

schema_template

spyglass_workshop.schema_template

Workshop schema template.

This module defines a minimal Spyglass pipeline with four tables:

  • MyParams — parameter lookup table
  • MyAnalysisSelection — staging table pairing data with parameters
  • MyAnalysis — computed analysis table
  • MyAnalysis.MyPart — part table storing per-iteration results

Each table foreign-key references :class:spyglass.common.Subject, which is pre-populated in the workshop MySQL instance.

Usage

Import this module in the Session 2 notebook to register the tables under your personal schema (workshop_<your-username>). Then follow the notebook instructions to populate and query them.

Exercise

Add a mean_result : float secondary field to MyAnalysis that stores the mean of all MyPart.result values for that key. Update make accordingly and re-run MyAnalysis().populate().

MyParams

Bases: SpyglassMixin, Lookup

Analysis parameter sets.

Each row defines one named configuration for MyAnalysis.make. The params blob stores a dictionary of keyword arguments passed directly to the analysis function.

Source code in src/spyglass_workshop/schema_template.py 
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
@schema
class MyParams(SpyglassMixin, dj.Lookup):
    """Analysis parameter sets.

    Each row defines one named configuration for ``MyAnalysis.make``.
    The ``params`` blob stores a dictionary of keyword arguments passed
    directly to the analysis function.
    """

    definition = """
    param_name : varchar(32)   # short descriptive name
    ---
    params     : blob          # dict of analysis parameters
    """
    contents = [
        ["default", {"n_iterations": 5, "scale": 1.0}],
        ["quick", {"n_iterations": 2, "scale": 0.5}],
    ]

    @classmethod
    def insert_default(cls):
        """Insert the default parameter sets defined in ``contents``."""
        cls().insert(cls.contents, skip_duplicates=True)

insert_default() classmethod

Insert the default parameter sets defined in contents.

Source code in src/spyglass_workshop/schema_template.py 
54
55
56
57
@classmethod
def insert_default(cls):
    """Insert the default parameter sets defined in ``contents``."""
    cls().insert(cls.contents, skip_duplicates=True)

MyAnalysisSelection

Bases: SpyglassMixin, Manual

Pairs of subjects and parameter sets to be analyzed.

Insert rows here to schedule work for MyAnalysis().populate(). Only the combinations you insert will be processed.

Source code in src/spyglass_workshop/schema_template.py 
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
@schema
class MyAnalysisSelection(SpyglassMixin, dj.Manual):
    """Pairs of subjects and parameter sets to be analyzed.

    Insert rows here to schedule work for ``MyAnalysis().populate()``.
    Only the combinations you insert will be processed.
    """

    definition = """
    -> Subject
    -> MyParams
    """

    @classmethod
    def insert_all(cls, param_name: str = "default"):
        """Insert every subject paired with the given parameter set.

        Parameters
        ----------
        param_name : str
            Name of the parameter set to pair with each subject.
        """
        cls().insert(
            [{**key, "param_name": param_name} for key in Subject.fetch("KEY")],
            skip_duplicates=True,
        )

insert_all(param_name='default') classmethod

Insert every subject paired with the given parameter set.

Parameters:

Name Type Description Default
param_name str

Name of the parameter set to pair with each subject.

 'default'
Source code in src/spyglass_workshop/schema_template.py 
73
74
75
76
77
78
79
80
81
82
83
84
85
@classmethod
def insert_all(cls, param_name: str = "default"):
    """Insert every subject paired with the given parameter set.

    Parameters
    ----------
    param_name : str
        Name of the parameter set to pair with each subject.
    """
    cls().insert(
        [{**key, "param_name": param_name} for key in Subject.fetch("KEY")],
        skip_duplicates=True,
    )

MyAnalysis

Bases: SpyglassMixin, Computed

Computed analysis results.

Populated automatically by MyAnalysis().populate(), which calls make for every unprocessed row in MyAnalysisSelection.

Source code in src/spyglass_workshop/schema_template.py 
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
@schema
class MyAnalysis(SpyglassMixin, dj.Computed):
    """Computed analysis results.

    Populated automatically by ``MyAnalysis().populate()``, which calls
    ``make`` for every unprocessed row in ``MyAnalysisSelection``.
    """

    definition = """
    -> MyAnalysisSelection
    ---
    total_result : int   # sum of all part results
    """

    class MyPart(SpyglassMixinPart, dj.Part):
        """Per-iteration results.

        One row per iteration of the analysis loop defined in
        ``MyAnalysis.make``.
        """

        definition = """
        -> MyAnalysis
        iteration : smallint unsigned   # iteration index (0-based)
        ---
        result    : int                 # result for this iteration
        """

    def make(self, key: dict):
        """Populate one row of ``MyAnalysis`` and its part table.

        Parameters
        ----------
        key : dict
            Primary key dict provided by ``populate``.  Contains
            ``subject_id`` and ``param_name``.
        """
        params = (MyParams & key).fetch1("params")
        n_iterations: int = params.get("n_iterations", 5)
        scale: float = params.get("scale", 1.0)

        part_rows = []
        total = 0
        for i in range(n_iterations):
            # ------------------------------------------------------------------
            # TODO (exercise): replace the placeholder below with a real
            # computation that uses ``key`` and ``scale``.
            # ------------------------------------------------------------------
            result = int(i * scale * 10)
            total += result
            part_rows.append({**key, "iteration": i, "result": result})

        # Insert the master row first, then parts (DataJoint requirement).
        self.insert1({**key, "total_result": total})
        self.MyPart().insert(part_rows)

    @staticmethod
    def summarize(key: str | dict) -> dict:
        """Return a summary dict for a given key.

        Parameters
        ----------
        key : str or dict
            Restriction identifying one ``MyAnalysis`` row.

        Returns
        -------
        dict
            Keys: ``subject_id``, ``param_name``, ``total_result``,
            ``n_parts``.
        """
        row = (MyAnalysis & key).fetch1()
        n_parts = len(MyAnalysis.MyPart & key)
        return {**row, "n_parts": n_parts}

MyPart

Bases: SpyglassMixinPart, Part

Per-iteration results.

One row per iteration of the analysis loop defined in MyAnalysis.make.

Source code in src/spyglass_workshop/schema_template.py 
102
103
104
105
106
107
108
109
110
111
112
113
114
class MyPart(SpyglassMixinPart, dj.Part):
    """Per-iteration results.

    One row per iteration of the analysis loop defined in
    ``MyAnalysis.make``.
    """

    definition = """
    -> MyAnalysis
    iteration : smallint unsigned   # iteration index (0-based)
    ---
    result    : int                 # result for this iteration
    """

make(key)

Populate one row of MyAnalysis and its part table.

Parameters:

Name Type Description Default
key dict

Primary key dict provided by populate. Contains subject_id and param_name.

 required
Source code in src/spyglass_workshop/schema_template.py 
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
def make(self, key: dict):
    """Populate one row of ``MyAnalysis`` and its part table.

    Parameters
    ----------
    key : dict
        Primary key dict provided by ``populate``.  Contains
        ``subject_id`` and ``param_name``.
    """
    params = (MyParams & key).fetch1("params")
    n_iterations: int = params.get("n_iterations", 5)
    scale: float = params.get("scale", 1.0)

    part_rows = []
    total = 0
    for i in range(n_iterations):
        # ------------------------------------------------------------------
        # TODO (exercise): replace the placeholder below with a real
        # computation that uses ``key`` and ``scale``.
        # ------------------------------------------------------------------
        result = int(i * scale * 10)
        total += result
        part_rows.append({**key, "iteration": i, "result": result})

    # Insert the master row first, then parts (DataJoint requirement).
    self.insert1({**key, "total_result": total})
    self.MyPart().insert(part_rows)

summarize(key) staticmethod

Return a summary dict for a given key.

Parameters:

Name Type Description Default
key str or dict

Restriction identifying one MyAnalysis row.

 required

Returns:

Type Description
dict

Keys: subject_id, param_name, total_result, n_parts.
Source code in src/spyglass_workshop/schema_template.py 
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
@staticmethod
def summarize(key: str | dict) -> dict:
    """Return a summary dict for a given key.

    Parameters
    ----------
    key : str or dict
        Restriction identifying one ``MyAnalysis`` row.

    Returns
    -------
    dict
        Keys: ``subject_id``, ``param_name``, ``total_result``,
        ``n_parts``.
    """
    row = (MyAnalysis & key).fetch1()
    n_parts = len(MyAnalysis.MyPart & key)
    return {**row, "n_parts": n_parts}