moved code to github

.gitignore

@@ -0,0 +1,112 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+.static_storage/
+.media/
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# idea
+*.iml
+.idea
+
+# manually added
+*~
+*#

LICENSE-CCBY

@@ -0,0 +1,99 @@
+Creative Commons Attribution 4.0 International Public License
+
+By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions.
+
+Section 1 – Definitions.
+
+    Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image.
+    Adapter's License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.
+    Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights.
+    Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements.
+    Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material.
+    Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License.
+    Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license.
+    Licensor means the individual(s) or entity(ies) granting rights under this Public License.
+    Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them.
+    Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world.
+    You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning.
+
+Section 2 – Scope.
+
+    License grant.
+        Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to:
+            reproduce and Share the Licensed Material, in whole or in part; and
+            produce, reproduce, and Share Adapted Material.
+        Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions.
+        Term. The term of this Public License is specified in Section 6(a).
+        Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material.
+        Downstream recipients.
+            Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License.
+            No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material.
+        No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i).
+
+    Other rights.
+        Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise.
+        Patent and trademark rights are not licensed under this Public License.
+        To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties.
+
+Section 3 – License Conditions.
+
+Your exercise of the Licensed Rights is expressly made subject to the following conditions.
+
+    Attribution.
+
+        If You Share the Licensed Material (including in modified form), You must:
+            retain the following if it is supplied by the Licensor with the Licensed Material:
+                identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);
+                a copyright notice;
+                a notice that refers to this Public License;
+                a notice that refers to the disclaimer of warranties;
+                a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
+            indicate if You modified the Licensed Material and retain an indication of any previous modifications; and
+            indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License.
+        You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information.
+        If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable.
+        If You Share Adapted Material You produce, the Adapter's License You apply must not prevent recipients of the Adapted Material from complying with this Public License.
+
+Section 4 – Sui Generis Database Rights.
+
+Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material:
+
+    for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database;
+    if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material; and
+    You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database.
+
+For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights.
+
+Section 5 – Disclaimer of Warranties and Limitation of Liability.
+
+    Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.
+    To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.
+
+    The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.
+
+Section 6 – Term and Termination.
+
+    This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.
+
+    Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:
+        automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
+        upon express reinstatement by the Licensor.
+    For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.
+    For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License.
+    Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
+
+Section 7 – Other Terms and Conditions.
+
+    The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed.
+    Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License.
+
+Section 8 – Interpretation.
+
+    For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License.
+    To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions.
+    No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.
+    Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.
+
+Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” The text of the Creative Commons public licenses is dedicated to the public domain under the CC0 Public Domain Dedication. Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at creativecommons.org/policies, Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses.
+
+Creative Commons may be contacted at creativecommons.org.

README.md

@@ -0,0 +1,23 @@
+# Welcome to the repository containing the python implementation of VIEW
+VIEW is a general purpose tool to analyze calcium imaging data of responses to sensory stimuli. 
+Its philosophy is two-dimensional.
+
+Dimension one - data handling: Data is treated in five steps. 
+
+1. load data, 
+2. calculate signals - e.g. ratio, movement or bleach correction etc.
+3. apply corrections using another data set - e.g. subtract air measurement
+4. create additional information - e.g. a map of localities for each glomerulus
+5. report about the data in two ways:
+    1. analyze data and generate output, interactively - e.g. time traces or false-color coded images.
+    2. analyze data and generate output, offline - using a list that controls which datasets to analyze.
+
+Importantly, 5.1 and 5.2 use exactly the same code. 
+Each of these steps offers a variety of options controlled by "flags" (e.g. different data formats, movement corrections, definitions for glomerulus location, format of output data). The sequential nature means new modules can be implemented easily.
+ 
+Dimension two - data analysis process: the user has interactive access to the data in order to evaluate it, and interactive access to all flag values, allowing the user to select the appropriate settings. Once the flags are chosen, the user generates a list of all measurements to be evaluated, and thus analyzes the entire dataset with identical settings, allowing for powerful (and correct) statistical analysis. 
+
+VIEW is based on a previous version written in IDL, mostly by Giovanni (1996-2015) with the help of many people, and is now written in Python, mostly by Ajay (2019), again with the help of many people, and including modules from various sources.
+
+
+For more information about installing and using VIEW, consult the [wiki](https://git.uni-konstanz.de/galizia/idl_python_translate/wikis/home)

VIEWtemplates/create_measurement_list_template.py

@@ -0,0 +1,186 @@
+from view.python_core.measurement_list import MeasurementList
+from view.python_core.measurement_list.importers import get_importer_class
+from view.python_core.flags import FlagsManager
+from collections import OrderedDict
+import pandas as pd
+import logging
+
+logging.basicConfig(level=logging.INFO)
+
+# ------------------- Some parameters about experimental setup, data structure and output file type --------------------
+# 3 for single wavelength Till Photonics Measurements
+# 4 for two wavelength Till Photonics Measurements
+# 20 for Zeiss Confocal Measurements
+LE_loadExp = 3
+
+# Mother of all Folders of your dataset
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_MotherOfAllFolders = r""
+
+# path of the "Data" folder in VIEW organization containing the data
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_Datapath = r""
+
+# path of the "Lists" folder in VIEW organization containing the list files
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_OdorInfoPath = r""
+
+# Choose measurement list output extension among ".lst", ".lst.xlsx", ".settings.xlsx"
+# VIEW does not support writing .xls list files anymore (nonetheless, it can read them and revise/update them to .xlsx)
+measurement_output_extension = ".lst.xlsx"
+
+# ------------------- A dictionary containing default values for metadata.----------------------------------------------
+# ------------------- Only metadata included in this dictionary will be written ----------------------------------------
+# ----Note that columns of the output measeurement list files will have the same order as below.------------------------
+
+default_values = OrderedDict()
+
+default_values['Measu'] = 0  # unique identifier for each line, corresponds to item in TILL photonics log file
+
+default_values['Label'] = "none"
+default_values['Odour'] = 'odor?'  # stimulus name, maybe extracted from label in the function "custom_func" below
+default_values['OConc'] = 0  # odor concentration, maybe extracted from label in the function "custom_func" below
+default_values['Analyze'] = -1  # whether to analyze in VIEWoff. Default 1
+
+default_values['Cycle'] = 0  # how many ms per frame
+default_values['DBB1'] = 'none'  # file name of raw data
+default_values['UTC'] = 0  # recording time, extracted from file
+
+default_values['PxSzX'] = '4.6'  # um per pixel, 1.5625 for 50x air objective, measured by Hanna Schnell July 2017 on Till vision system, with a binning of 8
+default_values['PxSzY'] = '4.6'  # um per pixel, 1.5625 for 50x air objective, measured by Hanna Schnell July 2017 on Till vision system, with a binning of 8
+
+default_values['Lambda'] = 0  # wavelength of stimulus. In TILL, from .log file, In Zeiss LSM, from .lsm file
+
+# These will be automatically filed for LE_loadExp=4
+default_values['dbb2'] = 'none'  # file name of raw data in dual wavelength recordings (FURA)
+# To include more columns, uncomment entries below and specify a default value.
+# #
+# block for first stimulus
+# default_values['StimON'] = -1  # stimulus onset, unit: frames, count starts at frame 1.
+# default_values['StimOFF'] = -1  # stimulus offset, unit: frames, count starts at frame 1.
+# default_values['StimLen'] = 0  # stimulus onset in ms from beginning - alternative to StimON
+# default_values['StimONms'] = -1  # stimulus length in ms - alternative to StimOFF
+# #
+# block for second stimulus
+# default_values['Stim2ON'] = 0  # stimulus onset, unit: frames, count starts at frame 1.
+# default_values['Stim2OFF'] = 0  # stimulus offset, unit: frames, count starts at frame 1.
+# default_values['Stim2Len'] = 0  # stimulus onset in ms from beginning - alternative to StimON
+# default_values['Stim2ONms'] = -1  # stimulus length in ms - alternative to StimOFF
+# #
+# default_values['Age'] = -1
+# default_values['Sex'] = 'o'
+# default_values['Side'] = 'none'
+# default_values['Comment'] = 'none'
+# #
+# default_values['MTime'] = 0
+# default_values['Control'] = 0
+# default_values['Pharma'] = 'none'
+# default_values['PhTime'] = 0
+# default_values['PhConc'] = 0
+# default_values['ShiftX'] = 0
+# default_values['ShiftY'] = 0
+# default_values['StimISI'] = 0
+# default_values['setting'] = 'none'
+# default_values['dbb3'] = 'none'
+# default_values['PosZ'] = 0
+# default_values['Countl'] = 0
+# default_values['slvFlip'] = 0
+# ----------------------------------------------------------------------------------------------------------------------
+
+# ----------------- A function used to modify list entries after automatic parsing of metadata -------------------------
+# ----------------- This function indicates what needs to be done for a row --------------------------------------------
+# ----------------- The same is internally applied to all rows of the measurement list----------------------------------
+
+
+def custom_func(list_row: pd.Series, animal_tag: str) -> pd.Series:
+
+    # Examples:
+    # list_row["StimON"] = 25
+    # list_row["Odour"] = get_odor_from_label(list_row["Label"])
+    # if list_row["Measu"]
+    # get Odor from another file based on the value of <animal_tag> and list_row["Label"]
+    return list_row
+
+# ----------------------------------------------------------------------------------------------------------------------
+
+# ------------------ A function defining the criteria for excluding measurements ---------------------------------------
+# ------------------ Currently applicable only for tillvision setups ---------------------------------------------------
+
+
+def measurement_filter(s):
+    # exclude blocks that have in the name "Snapshot" or "Delta"
+    # or that do not have any "_"
+    name = s["Label"]
+    label_not_okay = name.count('Snapshot') > 0 or name.count('Delta') > 0 or name.count('_') < 1
+    label_okay = not label_not_okay
+
+    # exclude blocks with less than two frames or no calibration
+    atleast_two_frames = False
+    if type(s["Timing_ms"]) is str:
+        if len(s["Timing_ms"].split(' ')) >= 2 and s["Timing_ms"] != "(No calibration available)":
+            atleast_two_frames = True
+
+    return label_okay and atleast_two_frames
+
+
+# ______________________________________________________________________________________________________________________
+
+
+# ------------------ names of columns that will be overwritten by old values -------------------------------------------
+# ------ these will only be used if a measurement list file with the same name as current output file exists -----------
+
+overwrite_old_values = ["Line", "PxSzX", "PxSzY", "Age", "Sex", "Prefer",
+                        "Comment", "Analyze", "Odour", "OConc"]
+
+# ______________________________________________________________________________________________________________________
+
+if __name__ == "__main__":
+
+    # initialize a FlagsManager object with values specified above
+    flags = FlagsManager()
+    flags.update_flags({"STG_MotherOfAllFolders": STG_MotherOfAllFolders,
+                        "STG_OdorInfoPath": STG_OdorInfoPath,
+                        "STG_Datapath": STG_Datapath})
+
+    # initialize importer
+    importer_class = get_importer_class(LE_loadExp)
+    importer = importer_class(default_values)
+
+    # open a dialog for choosing raw data files
+    # this returns a dictionary where keys are animal tags (STG_ReportTag) and
+    # values are lists of associated raw data files
+    animal_tag_raw_data_mapping = importer.ask_for_files(default_dir=flags["STG_Datapath"])
+    # make sure some files were chosen
+    assert len(animal_tag_raw_data_mapping) > 0, IOError("No files were chosen!")
+
+    for animal_tag, raw_data_files in animal_tag_raw_data_mapping.items():
+
+        # automatically parse metadata
+        metadata_df = importer.import_metadata(raw_data_files=raw_data_files,
+                                               measurement_filter=measurement_filter)
+        # inform user if no usable measurements were found
+        if metadata_df.shape[0] == 0:
+            logging.info(f"No usable measurements we found among the files "
+                         f"chosen for the animal {animal_tag}. Not creating a list file")
+        else:
+            # create a new Measurement list object from parsed metadata
+            measurement_list = MeasurementList.create_from_df(LE_loadExp=LE_loadExp,
+                                                              df=metadata_df)
+
+            # apply custom modifications
+            measurement_list.update_from_custom_func(custom_func=custom_func, animal_tag=animal_tag)
+
+            # set anaylze to 0 if raw data files don't exist
+            flags.update_flags({"STG_ReportTag": animal_tag})
+            measurement_list.sanitize(flags=flags,
+                                      data_file_extensions=importer.movie_data_extensions)
+
+            # construct the name of the output file
+            out_file = f"{flags.get_lst_file_stem()}{measurement_output_extension}"
+
+            # write measurement file to list
+            measurement_list.write_to_list_file(lst_fle=out_file, columns2write=default_values.keys(),
+                                                overwrite_old_values=overwrite_old_values)
+
+
+

VIEWtemplates/create_tapestry_template.py

@@ -0,0 +1,51 @@
+import view
+
+# this tells view all settings including the folder structure of your project
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+ymlfile = r""
+
+# please enter the paths of tapestry configuration files in ABSOLUTE form.
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+tapestry_config_files = [
+    "",
+    ""
+]
+
+
+# define a function that takes a row of the measurement list and returns a string. This string will be placed
+# below the overview
+def text_below(row):
+    # Example 1: the label below each overview will just be the odor
+    return row['Odour']
+
+    # Example 2: the label below each overview will be the odor and concentration separated by an underscore ("_")
+    # return f"{row['Odour']}_{row['OConc']}"
+
+
+# define a function that takes a row of the measurement list and returns a string. This string will be placed
+# next to the overview on the top right.
+def text_right_top(row):
+    pass
+
+
+# define a function that takes a row of the measurement list and creates the string to be placed below the overview
+def text_right_bottom(row):
+    pass
+
+
+if __name__ == '__main__':
+
+    for tapestry_config_file in tapestry_config_files:
+
+        # When text_right_bottom_func and text_right_top_func are specified to be None, the upper and lower limits
+        # of the data shown in an overview will be printed to its right top and right bottom side
+        # To customize the strings printed there, please define the functions 'text_right_top' and
+        # 'text_right_bottom above and pass them appropriately below, in the place on 'None'.
+        html_out_file, view_obj = view.create_tapestry(init_yml_flags_file=ymlfile,
+                                                       tapestry_config_file=tapestry_config_file,
+                                                       text_below_func=text_below,
+                                                       text_right_top_func=None,
+                                                       text_right_bottom_func=None)
+
+        # backup this script and the yml file used next to the created tapestries
+        view_obj.backup_script_flags_configs_for_tapestries(files=[__file__, ymlfile, tapestry_config_file])

VIEWtemplates/export_movies_single_animal.py

@@ -0,0 +1,77 @@
+import view
+
+# this tells view all settings including the folder structure of your project
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+ymlfile = r""
+
+# any  manual changes to flags, add to dictionary as required
+flags_to_update = {
+    ## Example:
+    # "CTV_scalebar": True,
+    # "mv_xgap": 30,
+    # "mv_ygap": 30,
+    # "mv_exportFormat": "stack_tif",
+    # .....
+}
+
+# specify the animal to use
+animal = ...
+
+# specify the measus for which movies are to be generated and corresponding flag value changes
+measu_flags_dict = {
+    ## Example
+    # 34:
+    #     {
+    #      "mv_FirstFrame": 40,
+    #      "mv_LastFrame": 160
+    #     },
+    # 36:
+    #     {
+    #      "mv_FirstFrame": 30,
+    #      "mv_LastFrame": 140
+    #     },
+    # 50:
+    #     {
+    #      "mv_FirstFrame": 160,
+    #      "mv_LastFrame": 270
+    #     },
+    # 56: {},
+    # 58:
+    #     {
+    #       "mv_FirstFrame": 80,
+    #       "mv_LastFrame": 260
+    #     }
+}
+
+if __name__ == '__main__':
+
+    # iterate over measurements of the animal
+    for measu, measu_flags in measu_flags_dict.items():
+
+        # create a view object
+        view_obj = view.VIEW()
+
+        # load flags from yml file
+        view_obj.update_flags_from_ymlfile(ymlfile)
+
+        # update flags specified locally
+        view_obj.update_flags(flags_to_update)
+
+        # initialize view object with animal
+        view_obj.initialize_animal(animal=animal)
+
+        # load a measurement for the animal
+        view_obj.load_measurement_data_from_current_animal(measu)
+
+        # calculate signals
+        view_obj.calculate_signals()
+
+        # update movie flags for this measu
+        view_obj.update_flags(measu_flags)
+
+        # save movie for the loaded data
+        view_obj.export_movie_for_current_measurement()
+
+    if len(measu_flags_dict):
+        # backup this script and the yml file used next to the created GDMs
+        view_obj.backup_script_flags_configs_for_movies(files=[__file__, ymlfile])

VIEWtemplates/export_movies_template.py

@@ -0,0 +1,53 @@
+import view
+
+# this tells view all settings including the folder structure of your project
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+ymlfile = r""
+
+# any  manual changes to flags, add to dictionary as required
+flags_to_update = {
+    # Example:
+    # "CTV_scalebar": True,
+    # "x_gap": 30,
+    # "y_gap": 30,
+    # "mv_individualScale": 2,
+    # .....
+}
+
+# list of animals for which movies are to generated
+animals = [
+    "",
+    ""
+]
+
+if __name__ == '__main__':
+
+    # create a view object
+    view_obj = view.VIEW()
+
+    # load flags from yml file
+    view_obj.update_flags_from_ymlfile(ymlfile)
+
+    # update flags specified locally
+    view_obj.update_flags(flags_to_update)
+
+    # iterate over animals
+    for animal in animals:
+
+        # initialize view object with animal
+        view_obj.initialize_animal(animal=animal)
+
+        # iterate over measurements of the animal
+        for measu in view_obj.get_measus_for_current_animal(analyze_values_to_use=(1,)):
+
+            # load a measurement for the animal
+            view_obj.load_measurement_data_from_current_animal(measu)
+
+            # calculate signals
+            view_obj.calculate_signals()
+
+            # save movie for the loaded data
+            view_obj.export_movie_for_current_measurement()
+
+    # backup this script and the yml file used next to the created GDMs
+    view_obj.backup_script_flags_configs_for_movies(files=[__file__, ymlfile])

VIEWtemplates/export_processed_data_as_mat_files_template.py

@@ -0,0 +1,43 @@
+from view import VIEW
+from view.python_core.matfile import export_processed_data_as_mat_file
+
+# this tells view all settings including the folder structure of your project
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+ymlfile = r""
+
+# any  manual changes to flags, add to dictionary as required
+flags_to_update = {
+    # Example: flags for CTV specification
+}
+
+# list of animals for which MAT files are to be generated
+animals = [
+    "",
+    ""
+]
+
+if __name__ == '__main__':
+
+    # create a view object
+    view_obj = VIEW()
+
+    # load flags from yml file
+    view_obj.update_flags_from_ymlfile(ymlfile)
+
+    # update flags specified locally
+    view_obj.update_flags(flags_to_update)
+
+    # iterate over animals
+    for animal in animals:
+
+        # initialize view object with animal
+        view_obj.initialize_animal(animal=animal)
+
+        # export data as MAT file
+        export_processed_data_as_mat_file(view_obj, analyze_values_to_use=(1,))
+
+    # backup this script and the yml file used next to the created MAT files
+    view_obj.backup_script_flags_configs_for_processed_data_output(files=[__file__, ymlfile], format_name="MAT")
+
+
+

VIEWtemplates/export_traces_template.py

@@ -0,0 +1,74 @@
+import view
+import pandas as pd
+import logging
+import pathlib as pl
+
+# this tells view all settings including the folder structure of your project
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+from view.python_core.gdm_generation.gdm_data_classes import GDMFile
+
+ymlfile = r""
+
+# any  manual changes to flags, add to dictionary as required
+flags_to_update = {
+    "RM_ROITrace": 3,  # set to 0 for .coor files, 3 for .roi files and 4 for .roi.tif
+    # Other flag changes can be included, for example:
+    # CTV_scalebar: True,
+    # mv_individualScale: 2,
+    # .....
+}
+
+# list of animals for which traces are to be exported
+animals = [
+    "",
+    ""
+]
+
+if __name__ == '__main__':
+
+    # create a view object
+    view_obj = view.VIEW()
+
+    # load flags from yml file
+    view_obj.update_flags_from_ymlfile(ymlfile)
+
+    # update flags specified locally
+    view_obj.update_flags(flags_to_update)
+
+    # iterate over animals
+    for animal in animals:
+
+        # initialize view object with animal
+        view_obj.initialize_animal(animal=animal)
+
+        # get ROI information for this animal
+        roi_data_dict, roi_file = view_obj.get_roi_info_for_current_animal()
+
+        # initialize and empty data frame to accumulate data
+        gdm_file = GDMFile()
+
+        # iterate over measurements of the animal
+        for measu in view_obj.get_measus_for_current_animal(analyze_values_to_use=(1,)):
+
+            # load a measurement for the animal
+            view_obj.load_measurement_data_from_current_animal(measu)
+
+            # calculate signals
+            view_obj.calculate_signals()
+
+            # create glodatamix for the loaded measurement
+            gdm_file_this_measu, _ = view_obj.get_gdm_file_for_current_measurement(roi_data_dict)
+
+            # accumulate
+            gdm_file.append_from_a_gdm_file(gdm_file_this_measu)
+
+        # compose output file name and create parent directory if needed
+        output_file = view_obj.flags.get_gloDatamix_file_for_current_animal()
+        pl.Path(output_file).parent.mkdir(exist_ok=True)
+
+        # save gloDatamix file
+        gdm_file.write_to_csv(output_file)
+        logging.getLogger("VIEW").info(f"Wrote gloDatamix to {output_file}")
+
+        # backup this script and the yml file used next to the created GDMs
+        view_obj.backup_script_flags_configs_for_GDMs(files=[__file__, ymlfile])

VIEWtemplates/pro2tapestryConfigYML_template.py

@@ -0,0 +1,42 @@
+import pathlib as pl
+from view.idl_folder_translation.pro2tapestry_conf import parse_animal_tag, convert_pro_to_tapestry_config
+
+# input files are expected to be in this folder
+input_folder_containing_pros = r""
+
+# output files will be created in this folder
+output_folder_for_tapestry_configs = r""
+
+# just the name of the file, without the path
+# each of these files are expected to be found in the folder pointed to by `input_folder_containing_pros`
+pro_file_names = [
+    "",
+    ""
+]
+
+# these flags will override flags initializations from .pro files if present, else will be added.
+flags_to_override = {
+    "SO_individualScale": 3
+}
+
+
+if __name__ == '__main__':
+
+    # iterate through each of
+    for pro_file_name in pro_file_names:
+
+        # parse the animal tag from the file name of the .pro file
+        animal_tag = parse_animal_tag(pro_file_name)
+
+        # construct the full path of the .pro file
+        pro_file_path = pl.Path(input_folder_containing_pros) / pro_file_name
+
+        # construct the path for the output yml file
+        pro_file_name_stem = pro_file_name.split(".")[0]
+        op_yml_path = pl.Path(output_folder_for_tapestry_configs) / f"{pro_file_name_stem}.yml"
+
+        # convert
+        convert_pro_to_tapestry_config(
+            input_pro_file=pro_file_path, output_yml_file=op_yml_path, animal_tag=animal_tag,
+            flags_to_override=flags_to_override
+        )

VIEWtemplates/update_measurement_list_template.py

@@ -0,0 +1,119 @@
+from view.python_core.measurement_list import MeasurementList
+from view.python_core.flags import FlagsManager
+import easygui
+import pandas as pd
+import logging
+import pathlib as pl
+
+logging.basicConfig(level=logging.INFO)
+
+# ------------------- Some parameters about experimental setup, data structure and output file type --------------------
+# 3 for single wavelength Till Photonics Measurements
+# 4 for two wavelength Till Photonics Measurements
+# 20 for Zeiss Confocal Measurements
+LE_loadExp = 3
+
+# Mother of all Folders of your dataset
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_MotherOfAllFolders = r""
+
+# path of the "Data" folder in VIEW organization containing the data
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_Datapath = r""
+
+# path of the "Lists" folder in VIEW organization containing the list files
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_OdorInfoPath = r""
+
+# Choose measurement list output extension among ".lst", ".lst.xls", ".settings.xls"
+# please use ".lst.xls" when converting old list files
+measurement_output_extension = ".lst.xls"
+
+# ----------------------------------------------------------------------------------------------------------------------
+
+# ----------------- A function used to add new columns to the list file ------------------------------------------------
+# ----------------- This function indicates how to add new entries to a row --------------------------------------------
+# ----------------- possibly using other existing row values -----------------------------------------------------------
+# ----------------- The same logic is apply to all rows to create entire new columns -----------------------------------
+
+
+def custom_func(list_row: pd.Series, animal_tag: str) -> pd.Series:
+
+    # NOTE: take care when modifying column values that already exist. Old values will be lost!
+
+    # these changes are always required when updating a .LST file used with IDL
+    # ------------------------------------------------------------------------------------------------------------------
+    # update to comment to indicate that the list has been imported for use with pyVIEW
+    list_row["Comment"] += "_2pyVIEW"
+
+    stim_cols = ["StimON", "StimOFF", "Stim2ON", "Stim2OFF"]
+
+    # frames in IDL .LST files were numbered 1, 2, 3... In pyVIEW they are numbered 0, 1, 2....
+    for stim_col in stim_cols:
+
+        if stim_col in list_row:
+            list_row[stim_col] -= 1
+    # ------------------------------------------------------------------------------------------------------------------
+
+    # Examples:
+    # new_columns["Stim2ON"] = 25
+    # list_row["Odour"] = get_odor_from_label(list_row["Label"])
+    # if list_row["Measu"]
+    # get Odor from another file based on the value of <animal_tag> and list_row["Label"]
+
+    return list_row
+
+# ----------------------------------------------------------------------------------------------------------------------
+
+
+if __name__ == "__main__":
+
+    # initialize a FlagsManager object with values specified above
+    flags = FlagsManager()
+    flags.update_flags({"STG_MotherOfAllFolders": STG_MotherOfAllFolders,
+                        "STG_OdorInfoPath": STG_OdorInfoPath,
+                        "STG_Datapath": STG_Datapath})
+
+    # open a dialog for choosing existing list files
+    existing_measurement_list_files = easygui.fileopenbox(
+        msg="Choose one or more measurement list files to update", multiple=True,
+        default=f"{flags.get_list_dir_str()}/*"
+    )
+
+    # make sure some files were chosen
+    assert len(existing_measurement_list_files) > 0, IOError("No files were chosen!")
+
+    for existing_measurement_list_file in existing_measurement_list_files:
+
+        # create a measurement list object
+        measurement_list = MeasurementList.create_from_lst_file(
+            lst_fle=existing_measurement_list_file, LE_loadExp=LE_loadExp)
+
+        # parse animal tag from measurement list file name
+        animal_tag = measurement_list.get_STG_ReportTag()
+
+        # inform user if no usable measurements were found
+        if len(measurement_list.get_measus()) == 0:
+            logging.info(f"No usable measurements found in {existing_measurement_list_file}. "
+                         f"Not updating it.")
+        else:
+
+            # apply custom modifications
+            measurement_list.update_from_custom_func(custom_func=custom_func, animal_tag=animal_tag)
+
+            # set analyze to 0 if raw data files don't exist
+            flags.update_flags({"STG_ReportTag": animal_tag})
+            measurement_list.sanitize_based_on_loading(flags)
+
+            # create the output file name
+            existing_measurement_list_file_path = pl.Path(existing_measurement_list_file)
+            ml_name_stem = existing_measurement_list_file_path.name.split(".")[0]
+            output_file_path \
+                = existing_measurement_list_file_path.parent / f"{ml_name_stem}{measurement_output_extension}"
+
+            # write measurement file to list
+            measurement_list.write_to_lst_file_cross_format(
+                output_lst_file=str(output_file_path), backup_current_file=True
+                )
+
+            existing_measurement_list_file_path.unlink()

conda_env.yml

@@ -0,0 +1,14 @@
+name: view
+channels:
+  - defaults
+dependencies:
+  - python=3.7
+  - jinja2
+  - seaborn
+  - bokeh
+  - xlrd
+  - pyqtgraph
+  - whoosh
+  - openpyxl
+  - jupyter
+  - appdirs

conda_env_development.yml

@@ -0,0 +1,20 @@
+name: view_dev
+channels:
+  - defaults
+  - conda-forge
+dependencies:
+  - python=3.7
+  - jinja2
+  - easygui  # only available in conda-forge
+  - seaborn
+  - bokeh
+  - xlrd
+  - pyqtgraph
+  - whoosh
+  - moviepy  # only available on conda-forge
+  - openpyxl
+  - ipdb
+  - nose
+  - ipython
+  - jupyter
+  - appdirs

log2list_examples/create_measurement_list_VTK2021.py

@@ -0,0 +1,217 @@
+from view.python_core.measurement_list import MeasurementList
+from view.python_core.measurement_list.importers import get_importer_class
+from view.python_core.flags import FlagsManager
+from collections import OrderedDict
+import pandas as pd
+import logging
+
+logging.basicConfig(level=logging.INFO)
+
+# ------------------- Some parameters about experimental setup, data structure and output file type --------------------
+# 3 for single wavelength Till Photonics Measurements
+# 4 for two wavelength Till Photonics Measurements
+# 20 for Zeiss Confocal Measurements
+LE_loadExp = 3
+
+# Mother of all Folders of your dataset
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+#STG_STG_STG_MotherOfAllFolders = r"/home/ajay/Nextcloud/VTK_2021/bee/HS_210521_test"
+STG_STG_STG_MotherOfAllFolders = r"/Users/galizia/Documents/DATA/VTK_test/YT_VTK"
+#STG_STG_STG_MotherOfAllFolders = r"/Users/galizia/Documents/DATA/HS_210521_test"
+
+
+# path of the "Data" folder in VIEW organization containing the data
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_Datapath = r"01_DATA"
+
+# path of the "Lists" folder in VIEW organization containing the list files
+# On Windows, if you copy paths from the file explorer, make sure the string below is always of the form r"......"
+STG_OdorInfoPath = r"02_LISTS"
+
+# Choose measurement list output extension among ".lst", ".lst.xlsx", ".settings.xlsx"
+# VIEW does not support writing .xls list files anymore (nonetheless, it can read them and revise/update them to .xlsx)
+measurement_output_extension = ".lst.xlsx"
+
+# ------------------- A dictionary containing default values for metadata.----------------------------------------------
+# ------------------- Only metadata included in this dictionary will be written ----------------------------------------
+# ----Note that columns of the output measeurement list files will have the same order as below.------------------------
+
+default_values = OrderedDict()
+
+default_values['Measu'] = 0  # unique identifier for each line, corresponds to item in TILL photonics log file
+
+default_values['Label'] = "none"
+default_values['Odour'] = 'odor?'  # stimulus name, maybe extracted from label in the function "custom_func" below
+default_values['OConc'] = 0  # odor concentration, maybe extracted from label in the function "custom_func" below
+default_values['Analyze'] = -1  # whether to analyze in VIEWoff. Default 1
+
+default_values['Cycle'] = 0  # how many ms per frame
+default_values['DBB1'] = 'none'  # file name of raw data
+default_values['UTC'] = 0  # recording time, extracted from file
+
+default_values['PxSzX'] = '4.6'  # um per pixel, 1.5625 for 50x air objective, measured by Hanna Schnell July 2017 on Till vision system, with a binning of 8
+default_values['PxSzY'] = '4.6'  # um per pixel, 1.5625 for 50x air objective, measured by Hanna Schnell July 2017 on Till vision system, with a binning of 8
+
+default_values['Lambda'] = 0  # wavelength of stimulus. In TILL, from .log file, In Zeiss LSM, from .lsm file
+
+# These will be automatically filed for LE_loadExp=4
+default_values['dbb2'] = 'none'  # file name of raw data in dual wavelength recordings (FURA)
+# To include more columns, uncomment entries below and specify a default value.
+# #
+# block for first stimulus
+# default_values['StimON'] = -1  # stimulus onset, unit: frames, count starts at frame 1.
+# default_values['StimOFF'] = -1  # stimulus offset, unit: frames, count starts at frame 1.
+# default_values['StimLen'] = 0  # stimulus onset in ms from beginning - alternative to StimON
+# default_values['StimONms'] = -1  # stimulus length in ms - alternative to StimOFF
+# #
+# block for second stimulus
+# default_values['Stim2ON'] = 0  # stimulus onset, unit: frames, count starts at frame 1.
+# default_values['Stim2OFF'] = 0  # stimulus offset, unit: frames, count starts at frame 1.
+# default_values['Stim2Len'] = 0  # stimulus onset in ms from beginning - alternative to StimON
+# default_values['Stim2ONms'] = -1  # stimulus length in ms - alternative to StimOFF
+# #
+# default_values['Age'] = -1
+# default_values['Sex'] = 'o'
+# default_values['Side'] = 'none'
+# default_values['Comment'] = 'none'
+# #
+# default_values['MTime'] = 0
+# default_values['Control'] = 0
+# default_values['Pharma'] = 'none'
+# default_values['PhTime'] = 0
+# default_values['PhConc'] = 0
+# default_values['ShiftX'] = 0
+# default_values['ShiftY'] = 0
+# default_values['StimISI'] = 0
+# default_values['setting'] = 'none'
+# default_values['dbb3'] = 'none'
+# default_values['PosZ'] = 0
+# default_values['Countl'] = 0
+# default_values['slvFlip'] = 0
+# ----------------------------------------------------------------------------------------------------------------------
+
+# ----------------- A function used to modify list entries after automatic parsing of metadata -------------------------
+# ----------------- This function indicates what needs to be done for a row --------------------------------------------
+# ----------------- The same is internally applied to all rows of the measurement list----------------------------------
+
+def get_odorinfo_from_label(label):
+    # format for label is: concentration to the right, with a minus sign
+    # Odor next to it, separated by underscore
+    # IMPERARIVE: only ONE "-"
+
+    # is the information for a concentration present? Detect "-"
+    parts = label.split("-")
+    if len(parts) == 2:
+        concentration = "-" + parts[1] 
+    else:
+        concentration = '0'
+    # now look at the remaining part, the last bit is the odour
+    parts = parts[0]
+    if (parts[-1] == '_'): #remove trailing underscore
+        parts = parts[:-1]
+    parts = parts.split("_")
+    if len(parts) > 1:
+        odor = parts[-1] 
+    else:
+        odor = 'unknown'
+    return [odor, concentration]
+
+def custom_func(list_row: pd.Series, animal_tag: str) -> pd.Series:
+
+    list_row['StimON']   = 24
+    list_row['StimLen']  = 1000 
+    list_row['Stim2ON']  = 36
+    list_row['Stim2Len'] = 1000 
+    list_row['Comment']  = 'VTK2021'
+    list_row['Line']     = 'bee'
+    # Examples:
+    # list_row["StimON"] = 25
+    (list_row["Odour"],list_row["OConc"]) = get_odorinfo_from_label(list_row["Label"])
+    # if list_row["Measu"]
+    # get Odor from another file based on the value of <animal_tag> and list_row["Label"]
+    return list_row
+
+
+# ----------------------------------------------------------------------------------------------------------------------
+
+# ------------------ A function defining the criteria for excluding measurements ---------------------------------------
+# ------------------ Currently applicable only for tillvision setups ---------------------------------------------------
+
+
+def measurement_filter(s):
+    # exclude blocks that have in the name "Snapshot" or "Delta"
+    # or that do not have any "_"
+    name = s["Label"]
+    label_not_okay = name.count('Snapshot') > 0 or name.count('Delta') > 0 or name.count('_') < 1
+    label_okay = not label_not_okay
+
+    # exclude blocks with less than two frames or no calibration
+    atleast_two_frames = False
+    if type(s["Timing_ms"]) is str:
+        if len(s["Timing_ms"].split(' ')) >= 2 and s["Timing_ms"] != "(No calibration available)":
+            atleast_two_frames = True
+
+    return label_okay and atleast_two_frames
+
+
+# ______________________________________________________________________________________________________________________
+
+
+# ------------------ names of columns that will be overwritten by old values -------------------------------------------
+# ------ these will only be used if a measurement list file with the same name as current output file exists -----------
+
+overwrite_old_values = ["Line", "PxSzX", "PxSzY", "Age", "Sex", "Prefer",
+                        "Comment", "Analyze", "Odour", "OConc"]
+
+# ______________________________________________________________________________________________________________________
+
+if __name__ == "__main__":
+
+    # initialize a FlagsManager object with values specified above
+    flags = FlagsManager()
+    flags.update_flags({"STG_STG_STG_MotherOfAllFolders": STG_STG_STG_MotherOfAllFolders,
+                        "STG_OdorInfoPath": STG_OdorInfoPath,
+                        "STG_Datapath": STG_Datapath})
+
+    # initialize importer
+    importer_class = get_importer_class(LE_loadExp)
+    importer = importer_class(default_values)
+
+    # open a dialog for choosing raw data files
+    # this returns a dictionary where keys are animal tags (STG_ReportTag) and
+    # values are lists of associated raw data files
+    animal_tag_raw_data_mapping = importer.ask_for_files(default_dir=flags["STG_Datapath"])
+    # make sure some files were chosen
+    assert len(animal_tag_raw_data_mapping) > 0, IOError("No files were chosen!")
+
+    for animal_tag, raw_data_files in animal_tag_raw_data_mapping.items():
+
+        # automatically parse metadata
+        metadata_df = importer.import_metadata(raw_data_files=raw_data_files,
+                                               measurement_filter=measurement_filter)
+        # inform user if no usable measurements were found
+        if metadata_df.shape[0] == 0:
+            logging.info(f"No usable measurements we found among the files "
+                         f"chosen for the animal {animal_tag}. Not creating a list file")
+        else:
+            # create a new Measurement list object from parsed metadata
+            measurement_list = MeasurementList.create_from_df(LE_loadExp=LE_loadExp,
+                                                              df=metadata_df)
+
+            # apply custom modifications
+            measurement_list.update_from_custom_func(custom_func=custom_func, animal_tag=animal_tag)
+
+            # set anaylze to 0 if raw data files don't exist
+            flags.update_flags({"STG_ReportTag": animal_tag})
+            measurement_list.sanitize(flags=flags,
+                                      data_file_extensions=importer.movie_data_extensions)
+
+            # construct the name of the output file
+            out_file = f"{flags.get_lst_file_stem()}{measurement_output_extension}"
+
+            # write measurement file to list
+            measurement_list.write_to_list_file(lst_fle=out_file, columns2write=default_values.keys(),
+                                                overwrite_old_values=overwrite_old_values)
+
+
+

log2list_examples/log2settings_VTK2021_old_for_reference.py

@@ -0,0 +1,492 @@
+# -*- coding: utf-8 -*-
+"""
+Program to read Till vision .log files
+and write .settings.csv files
+
+the program works like this (not all implemented yet):
+- set flag settings that are default values
+- read .log files and parse
+- extract information that is in there
+
+then, run Set_loca_Values_XXX, depending on reference_settings flag, e.g. 
+    - get FURA measurement partners
+    - set stimulus times
+    - etc (all the personal information)
+    - extract information from names
+
+Since default flag settings are peculiar to every user,
+and extracting information from names is too,
+this program should be modified by everybody. 
+
+Latest modification: 25.7.19 (Inga files)
+Output is .lst.xls, into folder /Lists
+
+"""
+
+import scipy as sp
+import datetime
+import pandas as pd
+import os
+import tkinter as tk
+from tkinter.filedialog import askopenfilenames
+from tillvisionio.vws import  VWSDataManager
+import pathlib as pl
+
+
+flag_OneDirectoryUp = True #if True, settings file is saved one directory up
+flag_intoListDirectory = True
+flag_writeMovies    = False # not used any more
+flag_outextension    = '.lst.xls'
+dbb_withDir = True  # extract dbbXX.pst together with last directory, e.g. '190301_locust_ip16.pst/dbb6C.pst'
+
+
+
+### set settings here!
+reference_settings = 'Temp' # 'Or42b_GC6' 'Or22a_GC6'
+## for Jibin: set values in function Set_Jibin_Values(lst_frame)
+
+
+class OldFileHandlerBlank(object):
+
+    def __init__(self):
+        super().__init__()
+        pass
+
+    def backup(self):
+
+        pass
+
+    def write_old_values(self, df, columns):
+
+        return df
+
+
+class OldFileHandler(OldFileHandlerBlank):
+
+    def __init__(self, lst_file):
+
+        super().__init__()
+
+        self.lst_fle = lst_file
+        self.old_lst_df = pd.read_excel(self.lst_fle).reset_index()
+        if "index" in self.old_lst_df.columns:
+            del self.old_lst_df["index"]
+        self.backup_path = None
+
+    def backup(self):
+
+        lst_fle_path = pl.Path(self.lst_fle)
+        self.backup_path = lst_fle_path.with_suffix(f".{datetime.datetime.now().strftime('%Y%m%d')}.xls")
+        self.old_lst_df.to_excel(str(self.backup_path))
+        print("Output file existed! Old file saved as ")
+        print(str(self.backup_path))
+
+    def write_old_values(self, df, columns: list):
+
+        if "Measu" not in columns:
+            columns.append("Measu")
+
+        mask = self.old_lst_df["Label"].apply(lambda x: x in df["Label"].values)
+
+        old_df_subset = self.old_lst_df.loc[mask, columns].set_index("Measu")
+
+        df_temp = df.set_index("Measu")
+
+        combined_df = df_temp.combine(old_df_subset, func=lambda s1, s2: s2, overwrite=False)
+
+        return combined_df.reset_index()
+
+
+def get_old_file_handler(lst_file):
+
+    if os.path.exists(lst_file):
+        return OldFileHandler(lst_file)
+    else:
+        return OldFileHandlerBlank()
+
+
+def Set_local_Values_Sercan(lst_frame):
+    lst_frame['StimON']   = 20
+    lst_frame['StimLen']  = 1000 
+    lst_frame['Stim2ON']  = 60
+    lst_frame['Stim2Len'] = 1000 
+    lst_frame['Comment'] = 'log2lst_Sercan_CalciumGreen'
+
+    lst_frame['Line']      = 'locust'
+    lst_frame['Age']       = '-1'
+    lst_frame['Sex']       = 'o'
+
+    #!!odor and concentration extracted from label!!
+    return lst_frame
+
+
+def get_default_line():
+    ################################################
+    ###   default values    ########################
+    ################################################
+
+    # if there is a function such as Set_Jibin_Values, it will overwrite some of these
+    default_comment = 'log2lst'
+    default_line = 'locust'
+    default_PxSzX = '1.5625'  # um per pixel, for 50x air objective, measured by Hanna Schnell July 2017, with a binning of 8
+    default_PxSzY = '1.5625'
+    default_age = '-1'
+    default_sex = 'o'
+    default_odor = 'unknown'
+    default_NOConc = '0'
+    default_setting = 'setting'
+    default_prefer = '1'  # which area from KNIME to plot
+    default_slvFlip = '0'
+
+    ################################################
+
+    # copy the default ones, some will be overwritten later
+
+    lst_line = pd.Series()
+
+    lst_line['Odour'] = default_odor
+    lst_line['OConc'] = default_NOConc
+    lst_line['setting'] = default_setting
+    lst_line['Comment'] = default_comment
+    lst_line['Line'] = default_line
+    lst_line['PxSzX'] = default_PxSzX
+    lst_line['PxSzY'] = default_PxSzY
+    lst_line['Age'] = default_age
+    lst_line['Sex'] = default_sex
+    lst_line['Prefer'] = default_prefer
+    lst_line["slvFlip"] = default_slvFlip
+
+    return lst_line
+
+
+def parse_label(label):
+
+    
+    if reference_settings == "Temp":
+        info = {}
+        info["Odour"] = 'not set'
+        info["OConc"] = 'not set'
+    else:
+        parts = label.split("_")
+        info = {}
+        info["Odour"] = parts[1]
+        info["OConc"] = int(parts[2])
+
+    return info
+
+
+
+def convert_vws_names_to_lst_names(vws_measurement_series, default_line):
+    """
+    Convert values from vws.log nomenclaure to .lst nomenclature
+    :param vws_measurement_series: pandas.Series
+    :return: pandas.series
+    """
+    lst_line = default_line.copy()
+    lst_line['Animal']    = vws_measurement_series["Label"]
+    lst_line['Measu']     = vws_measurement_series['index'] + 1
+    lst_line['Label']     = vws_measurement_series['Label']
+    lst_line['DBB1']      = vws_measurement_series["Location"]
+    lst_line['Cycle']     = vws_measurement_series["dt"]
+    lst_line['Lambda']    = vws_measurement_series['MonochromatorWL_nm']
+    lst_line['UTC']       = vws_measurement_series['UTCTime']
+    lst_line['StartTime'] = vws_measurement_series['StartTime']
+    lst_line["Analyze"] = vws_measurement_series["Analyze"]
+
+    #all others, just to not have empty columns
+    lst_line['dbb2']    = 'none'
+    lst_line['dbb3']    = 'none'
+    lst_line['MTime']    = 0
+    lst_line['Countl']    = 0
+    lst_line['Control']    = 0
+    lst_line['StimISI']    = 0
+    lst_line['PhConc']    = 0
+    lst_line['PhTime']    = 0
+    lst_line['Pharma']    = "no_pharma"
+    lst_line['PosZ']    = 0
+    lst_line['ShiftX']    = 0
+    lst_line['ShiftY']    = 0
+    lst_line['Stim2OFF']    = -1
+    lst_line['Stim2ON']    = -1
+    lst_line['StimOFF']    = -1
+    lst_line['StimON']    = -1
+
+    label_info = parse_label(vws_measurement_series["Label"])
+    lst_line["Odour"] = label_info["Odour"]
+    lst_line["OConc"] = label_info["OConc"]
+
+    return lst_line
+
+#####################################
+## DEFINE REFERENCE ODOR AND TIME!!!!
+#####################################
+
+
+
+def log2settings(in_logFile, out_trunc, animal):
+    """ converts a till photonics .vws.log into a lst. 
+    Function is not split into
+    read and write because that is never needed 
+    Many columns may not be needed, but are still defined for compatibility fear
+    """
+
+    # # some constants and declarations
+    # block_starts = []
+    # block_ends   = []
+    # block_names  = []
+    # flag_oldvalues = False
+    # in the future: replace with searching <end of info>
+    lst_labels = ["Measu","Label","Odour","DBB1", "Cycle","MTime","OConc","Control",
+                  "StimON","StimOFF","Pharma","PhTime","PhConc","Comment","ShiftX","ShiftY","StimISI",
+                  "setting","dbb2","dbb3","PxSzX","PxSzY","PosZ","Lambda","Countl", "slvFlip", "Stim2ON","Stim2OFF",
+                  "Age", "Analyze","UTC", "Animal"]
+    # last_time = 0
+    # if a settings file already exists, make a backup
+
+  
+#     # read log and parse
+#     with open(in_logFile, 'r') as fh:
+#         lines = [line.strip() for line in fh.readlines()]
+#     for i,line in enumerate(lines):
+#         match = re.search('^\[(.*)\]',line) # looks for lines with []
+#         if match:
+#             block_starts.append(i)
+#             block_names.append(match.group(1))
+#         match = re.search('end of info',line) # looks for lines with end of info
+#         if match:
+#             block_ends.append(i)
+#
+# # make a list of those blocks that follow the naming conventions
+#     valid_blocks = []
+#     # remove blocks that have in the name "Snapshot" or "Delta"
+#     # or that do not have any "_"
+#     # 'Fluo340nm_00' would pass
+#     for i,name in enumerate(block_names):
+#         if name.count('Snapshot') > 0 \
+#             or name.count('Delta') > 0 \
+#             or name.count('_') < 1:
+#             pass
+#         else:
+#             valid_blocks.append([i,block_starts[i],name,block_ends[i]])
+# # make a list for what is written in each block, with label. The log file has lines such as Date: 04/04/18
+#             # and this will move into e.g. {'Date ': ' 04/04/18'}
+#     Measurements = []
+#     for i,block_info in enumerate(valid_blocks):
+#         Measurements.append({'index':block_info[0],'label':block_info[2]})
+#         block = lines[block_info[1]+1:block_info[3]] # all the lines that belong to this block
+#         for line in block:
+#             line_split = line.split(':')
+#             if len(line_split) == 2:
+#                 key,value = line_split
+#             if len(line_split) > 2:
+#                 key = line_split[0]
+#                 value = ':'.join(line_split[1:]).strip()
+#             Measurements[i][key] = value
+#
+#     lst_frame  = pd.DataFrame(columns=lst_labels)
+#     for Measurement in Measurements:  # Measurement = Measurements[0]
+#         lst_line  = pd.DataFrame(columns=lst_labels) #one line for ease of writing
+#
+#         # analyze label given in TILL for odor and concentration - not used in Jibin
+# #        label_split = Measurement['label'].split('_')
+# #        Odour = default_odor
+# #        NOConc = default_NOConc
+# #        setting = default_setting
+# #        feedback = True
+# #        if len(label_split) == 4: ## e.g. GC_06_ETBE_-2
+# #            tmp, setting, Odour, NOConc = label_split
+# #            setting = tmp + '_' + setting #reassemble
+# #            feedback = False
+# #        if len(label_split) == 3: ## e.g. GC06_ETBE_-2
+# #            setting,Odour,NOConc = label_split
+# #            feedback = False
+# #        if len(label_split) == 2: ## eg. ETBE_-4
+# #            Odour,NOConc = label_split
+# #            feedback = False
+# #        if feedback:
+# #            print()
+# #            print("names in TILL should be of the type xxx_odor_conc, e.g. GC00_ETBE_-2")
+# #            print("or ETBE_-2 or GC_00_ETBE_-6")
+# #            print()
+#
+#         # time & analyze
+#         try:
+#             times = Measurement['timing [ms]'].strip()
+#             times = sp.array(times.split(' '),dtype='float64')
+#             # calculate frame rate as time of (last frame - first frame) / (frames-1)
+#             dt = str((times[-1]-times[0])/(len(times)-1))
+#             analyze = '1' # since there are at least two frames, and thus a time, I suppose it is worth analyzing
+#         except:
+#             dt = '-1'
+#             analyze = '0'
+#
+#         # location, check if pst, else -1
+#         ext = os.path.splitext(Measurement['Location'])[1]
+#         if ext != '.pst':
+#             Location = '-1'
+#         else:
+#             #tanimal = os.path.splitext(os.path.splitext(os.path.basename(fname))[0])[0] # tanimal
+#             dbb = os.path.splitext(Measurement['Location'].split('\\')[-1])[0] # dbb wo pst
+#             # Location = '\\'.join([tanimal + '.pst',dbb])  #use this line to add the animal's .pst directory
+#             Location = dbb #for \\Jibin\\JJ_01_C_F_180404a.pst\\dbbBA.pst, return 'dbbBA'
+# #    "MTime","Control","StimON","StimOFF","Pharma","PhTime","PhConc","ShiftX","ShiftY","StimISI","dbb2","dbb3","PosZ","Countl","slvFlip","Stim2ON","Stim2OFF",]
+
+    def measurement_filter(s):
+
+        # remove blocks that have in the name "Snapshot" or "Delta"
+        # or that do not have any "_"
+        name = s["Label"]
+        label_not_okay = name.count('Snapshot') > 0 or name.count('Delta') > 0 or name.count('_') < 1
+        label_okay = not label_not_okay
+
+        atleast_two_frames = False
+        if type(s["Timing_ms"]) is str:
+            if len(s["Timing_ms"].split(' ')) >= 2:
+                atleast_two_frames = True
+
+        return label_okay and atleast_two_frames
+
+    def additional_cols(s):
+
+        # time & analyze
+        try:
+            times = s['Timing_ms'].strip()
+            times = sp.array(times.split(' '), dtype='float64')
+            # calculate frame rate as time of (last frame - first frame) / (frames-1)
+            dt = str((times[-1]-times[0])/(len(times)-1))
+            analyze = '1' # since there are at least two frames, and thus a time, I suppose it is worth analyzing
+        except Exception as e:
+            dt = '-1'
+            analyze = '0'
+
+        # location, check if pst, else -1
+        ext = os.path.splitext(s['Location'])[1]
+        if ext != '.pst':
+            Location = '-1'
+        else:
+            # for "....\\Jibin\\JJ_01_C_F_180404a.pst\\dbbBA.pst", return 'JJ_01_C_F_180404a.pst\\dbbBA.pst'
+            dbb = pl.PureWindowsPath(s["Location"])
+            if dbb_withDir:
+                Location = str(pl.Path(dbb.parts[-2]) / dbb.stem)
+
+                out_trunc_path = pl.Path(out_trunc)
+                expected_path = out_trunc_path.parent.parent / "Data" / pl.Path(dbb.parts[-2]) / dbb.name
+                if os.path.isfile(str(expected_path)):
+                    print(f"File found at expected location: {str(expected_path)}")
+                else:
+                    print(f"File NOT found at expected location: {str(expected_path)}")
+
+            else:
+                Location = dbb.stem
+
+        return {"dt": dt, "Analyze": analyze, "Location": Location}
+
+    vws_manager = VWSDataManager(in_logFile)
+    measurements = vws_manager.get_all_metadata(filter=measurement_filter, additional_cols_func=additional_cols)
+
+    this_lst_frame = pd.DataFrame()
+
+    for measurement_index, measurement_row in measurements.iterrows():
+
+        lst_line = convert_vws_names_to_lst_names(vws_measurement_series=measurement_row,
+                                                  default_line=get_default_line())
+        this_lst_frame = this_lst_frame.append(lst_line, ignore_index=True)
+
+
+    labels_not_initialzed = set(lst_labels) - set(this_lst_frame)
+    assert labels_not_initialzed == set(), f"Some required columns were not initialized:\n{labels_not_initialzed}"
+
+    #     # if .settings existed before, maybe some entries were already given
+    #     # this assumes that the rows are the same!
+    #     # better: find row where old_lst_df.label == label
+    #     if flag_oldvalues:
+    #         this_lst_df = old_lst_df.loc[old_lst_df.Label == lst_line['label'],:] #till uses "label", I use "Label"
+    #         lst_line.loc[index,'Line']    = this_lst_df.iloc[0]["Line"]
+    #         # alternative, in one code line: line = old_lst_df.loc[old_lst_df.Label == Label].iloc[0]['line']
+    #         lst_line.loc[index,'PxSzX']   = this_lst_df.iloc[0]["PxSzX"]
+    #         lst_line.loc[index,'PxSzY']   = this_lst_df.iloc[0]["PxSzY"]
+    #         lst_line.loc[index,'Age']     = this_lst_df.iloc[0]["Age"]
+    #         lst_line.loc[index,'Sex']     = this_lst_df.iloc[0]["Sex"]
+    #         lst_line.loc[index,'Prefer']  = this_lst_df.iloc[0]["Prefer"]
+    #         lst_line.loc[index,'Comment'] = this_lst_df.iloc[0]["Comment"]
+    #         lst_line.loc[index,'Analyze'] = this_lst_df.iloc[0]["Analyze"]
+    #         #get old entry, for age, sex, line, comment.PxSzY, PxSzX, odorshift
+    #
+    #     # now collect all this into a dataframe, and move to next
+    #     lst_frame = lst_frame.append(lst_line)
+    #
+
+
+    return this_lst_frame
+# end function log2settings
+#now lst_frame contains all information
+        
+
+
+
+
+
+
+
+
+
+
+#######################################################################
+# MAIN starts here 
+#######################################################################
+# Choose raw files
+root = tk.Tk()
+root.withdraw() # so that windows closes after file chosen 
+root.attributes('-topmost', True)
+filenames = askopenfilenames(
+                parent=root,
+                title='Select one or more Till .log files',
+                filetypes=[('settings files', '*.log'), ('all files', '*')]
+                ) # ask user to choose file
+# i = 0 #for debugging       
+for i in range(len(filenames)):
+    in_logFile = filenames[i]
+    if flag_OneDirectoryUp:
+        #slowly. e.g. c:/me/myself/I/animaldir/animal.vws.log
+        lst_trunc = os.path.splitext(os.path.splitext(in_logFile)[0])[0] #remove two extensions
+        fn_tmp    = os.path.split(lst_trunc)[1] # name of the file (without extensions)
+        dir_oneup = os.path.split(os.path.split(lst_trunc)[0])[0] #path to parent directory
+        if flag_intoListDirectory:
+            out_trunc = os.path.join(dir_oneup,'Lists', fn_tmp) #add filename to "Lists" folder in path
+        else:
+            out_trunc = os.path.join(dir_oneup, fn_tmp) #add filename             
+        #result: 'c:/me/myself/I/animal'   extension, e.g. .lst.xls to be added
+    else:
+        out_trunc = os.path.splitext(os.path.splitext(in_logFile)[0])[0] # + '.settings' # removes both .vws and .log
+
+    animal = os.path.basename(out_trunc)
+
+    old_file_handler = get_old_file_handler(f"{out_trunc}{flag_outextension}")
+    old_file_handler.backup()
+
+    # read values from log file, into dataframe
+    lst_frame = log2settings(in_logFile, out_trunc, animal)
+    #'''''
+    #Outside the measurements loop. Set values according to a list, depends on reference_settings
+    #
+    # modify this and calculate additional things, depending on reference_settings
+    if reference_settings == 'Inga':
+        lst_frame = Set_local_Values_Sercan(lst_frame)
+#        lst_frame = CalcFuraFilms(lst_frame, out_trunc) # calculate and write a FURA film for each 340 in lst_frame
+
+    lst_frame = old_file_handler.write_old_values(lst_frame, ["Line", "PxSzX", "PxSzY", "Age", "Sex", "Prefer",
+                                                                   "Comment", "Analyze", "Odour", "OConc"])
+    
+    # write lst_frame
+    # write dataframe version to .xls file
+    print('writing to ',out_trunc,' and extension', flag_outextension)
+    if flag_outextension == '.lst.xls':
+        lst_frame.to_excel(out_trunc+'.lst.xls')  
+    if flag_outextension == '.lst':
+        lst_frame.to_csv(out_trunc+'.lst',sep='\t')
+    
+    # think about creating response overview image, and single-area glodatamix - copy from log2settings_valid
+    # write_martin_glodatamix(os.path.splitext(lst_fname)[0], lst_trunc, mask)
+    
+
+

setup.py

@@ -0,0 +1,51 @@
+from setuptools import setup, find_packages
+
+
+setup(
+    name="view",
+    use_scm_version=True,
+    setup_requires=['setuptools_scm'],
+    packages=find_packages(exclude=["^\."]),
+    package_data={
+        "view": [
+            "flags_and_metadata_definitions/*.csv",
+            "graphics/icons/*.svg",
+            "fonts/PixelOperator/*",
+            "fonts/OpenSans/*",
+            "jinja_templates/*",
+            "idl_folder_translation/*template.txt"
+        ]
+    },
+    install_requires=[
+        "pandas>=0.24.2",
+        "openpyxl>=3.0.3",
+        "scipy >=1.2.1",
+        "numpy>=1.16.3",
+        "matplotlib>=3.0.3",
+        "pyyaml>=3.3",
+        "Pillow>=6.0.0",
+        "moviepy>=1.0.0",
+        "scikit-image>=0.15.0",
+        "appdirs>=1.4.3",
+        # iltis ported to PyQt5 is needed, old iltis built using PyQt4 will not work.
+        # Source code available at https://gitlab.inf.uni-konstanz.de/ag-galizia/iltis.
+        # See wiki on https://gitlab.inf.uni-konstanz.de/ag-galizia/view for more info
+        "iltis@git+https://git@github.com/galizia-lab/ILTIS.git@1.4",
+        "tillvisionio@git+https://git@github.com/galizia-lab/tillvisionio.git@1.0",
+        "tifffile>=2019.7.26",
+        "easygui>=0.98.1",
+        "jinja2>=2.10.3",
+        "whoosh>=2.7.4",
+        "seaborn>=0.10.0",
+        "bokeh>=2.0.0",
+        "textfsm>=1.1.0",
+        "itables>=0.2.1",
+        "gekko>=0.2.8",
+        "quantities>=0.12.4",
+        "neo>=0.9.0"
+    ],
+
+    entry_points={"console_scripts": ["view = view.gui.start_view_gui:main"]},
+
+    python_requires=">=3.7"
+)

tests/common.py

@@ -0,0 +1,53 @@
+import pathlib as pl
+from view.gui.application_settings import get_view_qsettings_manager
+
+
+def get_example_data_root_path():
+
+    settings = get_view_qsettings_manager()
+    if settings.contains("view_test_data_path"):
+        existing_test_data_path_str = settings.value("view_test_data_path")
+        existing_test_data_path = pl.Path(existing_test_data_path_str)
+
+        if existing_test_data_path.is_dir():
+
+            return existing_test_data_path
+
+        else:
+            raise FileNotFoundError(
+                f"Could not find the following folder, to which VIEW is configured for storing test data."
+                f"\n\n{existing_test_data_path_str}.\n\nPlease run the script 'setup_testing.py' in the root "
+                f"directory of VIEW source code again to download view test data and configure view test path")
+
+    else:
+        raise ValueError(
+            "pyVIEW needs some data for testing. Please run the script 'setup_testing.py' in the root "
+            "directory of VIEW source code again to download view test data and configure view test path")
+
+
+def get_example_dataset_roots():
+    example_data_root = get_example_data_root_path()
+
+    dataset_roots = []
+    for child in example_data_root.iterdir():
+
+        if child.is_dir() and any(x.name.lower().find("list") > 0 or x.name.lower().find("settings") > 0
+                                  for x in child.iterdir()):
+
+            dataset_roots.append(child)
+
+    return dataset_roots
+
+
+def initialize_test_yml_list_measurement():
+
+    example_data_path = get_example_data_root_path()
+
+    example_dataset_moaf = example_data_path / "FakeData"
+    test_yml = example_dataset_moaf / "test_defaults.yml"
+    test_animal = "FakeData"
+    test_measu = 8
+
+    return str(test_yml), test_animal, test_measu
+
+

tests/export_movie_test.py

@@ -0,0 +1,403 @@
+from common import initialize_test_yml_list_measurement
+from view import VIEW
+import logging
+import pathlib as pl
+import shutil
+from nose.tools import raises
+
+
+def export_fake_data_movie(flags_to_update, movie_name_suffix):
+
+    test_yml, test_animal, test_measu = initialize_test_yml_list_measurement()
+
+    view = VIEW()
+
+    view.update_flags_from_ymlfile(test_yml)
+    view.update_flags(flags_to_update)
+    view.load_measurement_data(test_animal, test_measu)
+    view.calculate_signals()
+    op_filename = view.export_movie_for_current_measurement()
+
+    test_movies_folder = pl.Path(view.flags["STG_OdorReportPath"]) / "test_movies"
+
+    if not test_movies_folder.is_dir():
+        test_movies_folder.mkdir()
+
+    op_filepath = pl.Path(op_filename)
+    test_movie_path = test_movies_folder / f"{op_filepath.stem}{movie_name_suffix}{op_filepath.suffix}"
+
+    if test_movie_path.is_dir():
+        shutil.rmtree(test_movie_path)
+
+    op_filepath.replace(test_movie_path)
+
+
+def test_defaults():
+    """
+    Testing exporting movie with default flags
+    """
+
+    export_fake_data_movie({}, "defaults")
+
+
+def test_rotate_flags():
+    """
+    Testing export_movie with different rotate flags
+    """
+
+    for rot in range(1, 8):
+
+        flags = {"mv_rotateImage": rot}
+        export_fake_data_movie(flags, f"mv_rotateImage_{rot}")
+
+    flags = {"mv_reverseIt": True}
+    export_fake_data_movie(flags, "mv_reverseIt_True")
+
+
+def test_scale_flags():
+    """
+    Testing export_movie with different scale flags
+    """
+
+    percentile_value = 20
+    flags_with_percentile = {"mv_percentileScale": True, "mv_percentileValue": percentile_value}
+    flags_without_percentile = {}
+
+    cutborder = 5
+    flags_with_cutborder = {"mv_cutborder": cutborder}
+
+    flag_types = {f"_percentileValue{percentile_value}": flags_with_percentile,
+                  "": flags_without_percentile,
+                  f"_cutborder{cutborder}": flags_with_cutborder}
+
+    for label, flags_to_copy in flag_types.items():
+        for indiscale in [1, 2, 4, 5, 6,
+                          11, 12, 14, 15, 16,
+                          21, 22, 24, 25, 26]:
+            flags = flags_to_copy.copy()
+            flags["mv_individualScale"] = indiscale
+            export_fake_data_movie(flags, f"mv_individualScale{indiscale}{label}")
+
+        for indiscale in [3, 13, 23]:
+            flags = flags_to_copy.copy()
+            flags["mv_individualScale"] = indiscale
+            flags["mv_indiScale3factor"] = 0.25
+            export_fake_data_movie(flags, f"mv_individualScale{indiscale}_factor0p25{label}")
+
+            flags = flags_to_copy.copy()
+            flags["mv_individualScale"] = indiscale
+            flags["mv_indiScale3factor"] = 0.4
+            export_fake_data_movie(flags, f"mv_individualScale{indiscale}_factor0p4{label}")
+
+            flags = flags_to_copy.copy()
+            flags["mv_individualScale"] = indiscale
+            flags["mv_indiScale3factor"] = 0
+            export_fake_data_movie(flags, f"mv_individualScale{indiscale}_factor0{label}")
+
+
+def test_displayTime_flags():
+    """
+    Testing export movie with different flags for displaying frame time
+    """
+
+    export_fake_data_movie({"mv_displayTime": 0}, "without_frame_time")
+    export_fake_data_movie({"mv_displayTime": 0.8}, "with_frame_time")
+    export_fake_data_movie({"mv_displayTime": 0.5}, "with_frame_time_0p5")
+    export_fake_data_movie({"mv_displayTime": 0.25}, "with_frame_time_0p25")
+    export_fake_data_movie({"mv_displayTime": 0.8, "mv_suppressMilliseconds": True}, "with_frame_time_no_ms")
+
+
+def test_mark_stimulus_flags():
+    """
+    Testing export movie with different mark stimulus flags
+    """
+
+    possible_values = [0, 1, 2, 3, 21]
+
+    for ms in possible_values:
+        export_fake_data_movie({"mv_markStimulus": ms}, f"with_mark_stimulus_{ms}")
+
+
+def test_different_export_formats():
+    """
+    Testing export movie with different export formats
+    """
+
+    export_fake_data_movie({"mv_exportFormat": "libx264", 'mv_individualScale': 2}, "_indScale2_libx264")
+    export_fake_data_movie({"mv_exportFormat": "single_tif", 'mv_individualScale': 2}, "_indScale2")
+    export_fake_data_movie({"mv_exportFormat": "ayuv", 'mv_individualScale': 2}, "_indScale2_ayuv")
+
+def test_filters():
+    """
+    Testing export movie with temporal and spatial filters
+    """
+
+    export_fake_data_movie({"Signal_Signal_FilterSpaceFlag": True, "Signal_Signal_FilterSpaceSize": 3}, "space_filter_3")
+    export_fake_data_movie({"Signal_Signal_FilterTimeFlag": True, "Signal_Signal_FilterTimeSize": 3}, "time_filter_3")
+
+
+def test_cutters():
+    """
+    Testing export movie with temporal and spatial cutters
+    """
+
+    export_fake_data_movie({"mv_FirstFrame": 5, "mv_LastFrame": 75,
+                            'mv_individualScale': 3,
+                            "mv_indiScale3factor": 0.25
+                            }, "five_frame_cut_start_end")
+    export_fake_data_movie({"mv_cutborder": 5,
+                            'mv_individualScale': 3,
+                            "mv_indiScale3factor": 0.25
+                            }, "mv_cutborder_5")
+
+
+def test_correct_stimulus_onset():
+    """
+    Testing export movie with stimulus onset correction
+    """
+
+    export_fake_data_movie({"mv_correctStimulusOnset": 10, "mv_markStimulus": 1}, "mv_correct_stimulus_onset_10")
+    export_fake_data_movie({"mv_correctStimulusOnset": 1300, "mv_markStimulus": 1}, "mv_correct_stimulus_onset_1300")
+
+
+def test_thresholdOn():
+    """
+    Testing export movie with different values of mv_thresholdOn
+    """
+
+    threshold_on_vals = {"foto1": [("a1000", "a400"), ("r50", "r30")],
+                         "raw1": [("a1000", "a400"), ("r60", "r10")],
+                         "sig1": [("a0.75", "a0.65"), ("r60", "r10")]}
+
+    for within_area in (True, False):
+
+        for threshold_on, threshold_vals in threshold_on_vals.items():
+            for (threshold_pos, threshold_neg) in threshold_vals:
+
+                export_fake_data_movie({
+                                        'mv_withinArea': within_area,
+                                        'mv_thresholdOn': threshold_on,
+                                        'mv_lowerThreshPositiveResps': threshold_pos,
+                                        'mv_upperThreshNegativeResps': threshold_neg,
+                                        'mv_individualScale': 3,
+                                        "mv_indiScale3factor": 0.25},
+                                        f"mv_thresholdOn_{threshold_on}"
+                                        f"_vals_{threshold_pos}{threshold_neg}_withinArea_{within_area}")
+
+
+def test_thresholdShowImage():
+    """
+    Testing export movie with different settings for mv_thresholdShowImage
+    """
+
+    for threshold_show_image in ["foto1", "raw1", "bgColor"]:
+        for threshold_scale in ["full", "onlyShown"]:
+
+            export_fake_data_movie({"mv_thresholdShowImage": threshold_show_image,
+                                    "mv_thresholdScale": threshold_scale,
+                                    'mv_thresholdOn': "foto1",
+                                    'mv_lowerThreshPositiveResps': "a1000",
+                                    'mv_individualScale': 3,
+                                    "mv_indiScale3factor": 0.25
+                                    },
+                                    f"mv_thresholdOn_foto1_posVal_a1000_Image_"
+                                    f"{threshold_show_image}_scale_{threshold_scale}")
+
+def test_withinArea():
+    """
+    Testing export movie with mv_withinArea set
+    """
+
+    export_fake_data_movie({'mv_withinArea': True,
+                            'mv_individualScale': 3,
+                            "mv_indiScale3factor": 0.25},
+                           "mv_within_Mask_True")
+
+    export_fake_data_movie({'mv_withinArea': True,
+                            'mv_individualScale': 3,
+                            "mv_indiScale3factor": 0.25,
+                            "mv_cutborder": 5},
+                           "mv_within_Mask_True_cutBorder5")
+
+
+def test_bgColor():
+    """
+    Testing export movie with mv_bgColor set
+    """
+
+    export_fake_data_movie({"mv_bgColor": "g"},
+                           "mv_with_bgColor_green")
+
+
+def test_fgColor():
+    """
+    Testing export movie with mv_fgColor set
+    """
+
+    export_fake_data_movie({"mv_fgColor": "m"},
+                           "mv_with_fgColor_magenta")
+
+
+def test_mark_rois():
+    """
+    Testing export movie with ROIs marked
+    """
+
+    base_flags = {'mv_individualScale': 3, "mv_indiScale3factor": 0.25}
+
+    test_values = [10, 13, 14, 15]
+
+    for test_value in test_values:
+        flags2use = base_flags.copy()
+        flags2use["mv_showROIs"] = test_value
+        export_fake_data_movie(flags_to_update=flags2use, movie_name_suffix=f"mv_showROIs{test_value}")
+
+        flags2use_new = flags2use.copy()
+        flags2use_new["mv_rotateImage"] = 3
+        export_fake_data_movie(flags_to_update=flags2use_new, movie_name_suffix=f"mv_showROIs{test_value}_rotate3")
+
+        flags2use_new = flags2use.copy()
+        flags2use_new["mv_cutborder"] = 5
+        export_fake_data_movie(flags_to_update=flags2use_new, movie_name_suffix=f"mv_showROIs{test_value}_cutborder5")
+
+@raises(ValueError)
+def test_large_bordercut():
+    """
+    Testing export movie when mv_cutborder is inappropriately large
+    """
+
+    export_fake_data_movie({"mv_cutborder": 106}, "mv_impossible")
+
+
+def test_fonts():
+    """
+    Testing export movie with different fonts
+    """
+    export_fake_data_movie({"mv_markStimulus": 2, "mv_fontName": "DroidSerif-Bold"}, "mv_fontName_DroidSerifBold")
+    export_fake_data_movie({"mv_markStimulus": 2, "mv_fontName": "OpenSans-Regular"}, "mv_fontName_OpenSansRegular")
+    export_fake_data_movie({"mv_markStimulus": 2, "mv_fontName": "DejaVuSerif-Bold"}, "mv_fontName_DejaVuSerifBold")
+
+
+def test_mv_ygap():
+    """
+    Testing different settings of mv_ygap
+    """
+    for mv_ygap in [0, 10, 50, 100]:
+        export_fake_data_movie({"mv_ygap": mv_ygap}, f"mv_ygap_{mv_ygap}")
+
+
+def test_scale_legend_factor():
+    """
+    Testing setting mv_scaleLegendFactor
+    """
+
+    for factor in (10, 100):
+        export_fake_data_movie({"mv_individualScale": 2,
+                                "mv_percentileScale": True,
+                                "mv_percentileValue": 20,
+                                "mv_scaleLegendFactor": factor},
+                               f"_scaleLegendFactor{factor}")
+
+
+def test_bit_rate():
+    """
+    Testing setting mv_bitrate
+    """
+
+    export_fake_data_movie(
+        {
+            "mv_individualScale": 2,
+            "mv_bitrate": f"{12 * 1024}k",
+            "mv_exportFormat": "libx264"
+        },
+        "_bitrate_12M"
+    )
+
+# def test_with_recorded_data():
+#     """
+#     Testing view.python_core.movies.export_movie with recorded data
+#     :return:
+#     """
+#
+#     example_data_path = get_example_data_root_path()
+#
+#     test_list = example_data_path / "IP_Fura" / "IDLlist" / "190112_locust_ip.lst.xls"
+#     test_yml = example_data_path / "IP_Fura" / "usage_till.yml"
+#     full_output_name_without_extension = str(example_data_path / "IP_Fura" / "IDLoutput" / "movie_test")
+#     test_measu = 0
+#
+#     # test_list = example_data_path / "20190821_SetupB_Pixel_Calibration"/ "Lists" / "pixel_calibration.lst"
+#     # test_yml = example_data_path/ "20190821_SetupB_Pixel_Calibration"/ "calibration.yml"
+#     # full_output_name_without_extension = \
+#     #     example_data_path/"20190821_SetupB_Pixel_Calibration"/ "IDLoutput"/ "test_movie"
+#     # test_measu = 2
+#
+#
+#     # test_list = "/home/aj/SharedWithWindows/Sercan_CalciumGreen/Lists/20190809_SS_locust004_CaGreen.lst.xls"
+#     # test_yml = '/home/aj/SharedWithWindows/Sercan_CalciumGreen/usage_till.yml'
+#     # full_output_name_without_extension = "/home/aj/SharedWithWindows/Sercan_CalciumGreen/IDLoutput/movie"
+#     # test_measu = 0
+#
+#
+#     flags = FlagsManager()
+#     flags.read_flags_from_yml(str(test_yml))
+#
+#
+#     # flags.update_flags({"mv_exportFormat": "rawvideo"})
+#     # flags.update_flags({"mv_exportFormat": "libx264"})
+#     # flags.update_flags({"mv_bitrate": "2M"})
+#
+#     # flags.update_flags({"mv_exportFormat": "single_tif"})
+#
+#     flags.update_flags({"mv_exportFormat": "stack_tif"})
+#
+#     flags.update_flags({"mv_displayTime": True})
+#     # flags.update_flags({"mv_markStimulus": 0})
+#     # flags.update_flags({"mv_markStimulus": 1})
+#     flags.update_flags({"mv_markStimulus": 2})
+#
+#     flags.update_flags({"mv_individualScale": 2})
+#
+#     # flags.update_flags({"mv_individualScale": 3})
+#     # flags.update_flags({"mv_indiScale3factor": 0.2})
+#
+#     flags.update_flags({"mv_suppressMilliseconds": False})
+#
+#     flags.update_flags({"CTV_scalebar": True})
+#
+#     flags.update_flags({"mv_xgap": 60})
+#     flags.update_flags({"mv_ygap": 60})
+#
+#     # flags.update_flags({"mv_reverseIt": True})
+#
+#     # flags.update_flags({"mv_rotateImage": 7})
+#
+#     # flags.update_flags({"mv_SpeedFactor": 0.333})
+#
+#     # flags.update_flags({"mv_percentileScale": True})
+#     # flags.update_flags({"mv_percentileValue": 10})
+#
+#     measurement_list = MeasurementList(str(test_list), LE_loadExp="3")
+#
+#     p1 = measurement_list.get_p1_metadata_by_measu(test_measu)
+#
+#     loadDataMaster(p1=p1, flag=flags.to_series())
+#
+#     CalcSigMaster(p1=p1, flag=flags.to_series())
+#
+#     export_movie(p1, flags=flags,
+#                  full_filename_without_extension=full_output_name_without_extension)
+#
+
+
+if __name__ == "__main__":
+
+    logging.basicConfig(level=logging.INFO)
+    # test_with_recorded_data()
+    # test_scale_flags()
+    test_large_bordercut()
+    # test_withinArea()
+    # test_correct_stimulus_onset()
+    # test_mark_rois()
+    # test_different_export_formats()
+    # test_bit_rate()

tests/export_traces_test.py

@@ -0,0 +1,165 @@
+from common import initialize_test_yml_list_measurement
+from view import VIEW
+import pathlib as pl
+import shutil
+from view.python_core.ctvs import get_all_available_ctvs
+from view.python_core.gdm_generation.gdm_data_classes import GDMFile
+
+
+class TraceExporter(object):
+
+    def __init__(self):
+
+        super().__init__()
+        test_yml, self.test_animal, self.test_measu = initialize_test_yml_list_measurement()
+
+        self.view = VIEW()
+
+        self.view.update_flags_from_ymlfile(test_yml)
+
+    def load_and_export(self, flags_to_update, file_suffix, flags_suffix):
+
+        self.view.update_flags(flags_to_update)
+
+        self.view.initialize_animal(self.test_animal)
+
+        roi_data_dict, roi_file = self.view.get_roi_info_for_current_animal()
+
+        # initialize and empty data frame to accumulate data
+        gdm_file = GDMFile()
+
+        # iterate over measurements of the animal
+        for measu in self.view.get_measus_for_current_animal(analyze_values_to_use=(1,)):
+
+            # load a measurement for the animal
+            self.view.load_measurement_data_from_current_animal(measu)
+
+            # calculate signals
+            self.view.calculate_signals()
+
+            # create glodatamix for the loaded measurement
+            gdm_file_this_measu, _ = self.view.get_gdm_file_for_current_measurement(roi_data_dict)
+
+            # accumulate
+            gdm_file.append_from_a_gdm_file(gdm_file_this_measu)
+
+        # compose output file name
+        output_file = self.view.flags.get_gloDatamix_file_for_current_animal()
+
+        output_file_path = pl.Path(output_file)
+
+        test_gdm_folder =\
+            pl.Path(self.view.flags["STG_OdorReportPath"]) / "test_gdms" / \
+            f"{output_file_path.stem}{file_suffix}"
+
+        if not test_gdm_folder.is_dir():
+            test_gdm_folder.mkdir(parents=True)
+
+        test_output_file = test_gdm_folder / f"gdm{flags_suffix}{output_file_path.suffix}"
+
+        # save gloDatamix file
+        gdm_file.write_to_csv(test_output_file)
+
+
+def test_export_traces_rois():
+    """
+    Testing exporting traces using .roi files
+    """
+
+    exporter = TraceExporter()
+
+    coor_path = pl.Path(exporter.view.flags["STG_OdormaskPath"])
+    dest_roi_file = coor_path / "Fake_data.roi"
+
+    for fle in coor_path.iterdir():
+
+        if fle.name.startswith("FakeData") and fle.suffix == ".roi":
+
+            shutil.copy(str(fle), str(dest_roi_file))
+
+            exporter.load_and_export(
+                flags_to_update={"RM_ROITrace": 3},
+                file_suffix=f"_from_roi{fle.stem.lstrip('FakeData')}",
+                flags_suffix="_defaults"
+            )
+
+            dest_roi_file.unlink()
+
+
+def test_export_traces_mask_tif():
+    """
+    Testing exporting traces using .roi.tif files
+    """
+
+    exporter = TraceExporter()
+    exporter.load_and_export(
+        flags_to_update={"RM_ROITrace": 4},
+        file_suffix="_from_roi_tif",
+        flags_suffix="_defaults"
+    )
+
+
+def test_export_traces_different_ctvs():
+    """
+    Testing exporting traces with different CTVs
+    """
+
+    exporter = TraceExporter()
+    for ctv in get_all_available_ctvs():
+        exporter.load_and_export(
+            flags_to_update={"RM_ROITrace": 3, "CTV_Method": ctv},
+            file_suffix=f"_from_roi",
+            flags_suffix=f"_ctv{ctv}"
+        )
+
+
+def test_export_traces_within_ROI():
+    """
+    Testing exporting traces considering the area file
+    """
+
+    exporter = TraceExporter()
+    exporter.load_and_export(
+        flags_to_update={"RM_ROITrace": 3, "GDM_withinArea": True},
+        file_suffix="_from_roi",
+        flags_suffix="_withinArea_True"
+    )
+
+
+def test_export_traces_chunks_only():
+    """
+    Testing exporting traces considering the area file
+    """
+
+    exporter = TraceExporter()
+    exporter.load_and_export(
+        flags_to_update=
+        {
+            "RM_ROITrace": 3,
+            "GDM_outputType": "chunks_only",
+            "GDM_chunkPostStim": 2,  # in seconds
+            "GDM_chunkPreStim": 2,  # in seconds
+        },
+        file_suffix="_chunks_only",
+        flags_suffix="_2secPrePostStim"
+    )
+
+    exporter.load_and_export(
+        flags_to_update=
+        {
+            "RM_ROITrace": 3,
+            "GDM_outputType": "chunks_only",
+            "GDM_chunkPostStim": 100,  # in seconds
+            "GDM_chunkPreStim": 100,  # in seconds
+        },
+        file_suffix="_chunks_only",
+        flags_suffix="_full"
+    )
+
+
+if __name__ == '__main__':
+
+    test_export_traces_rois()
+    # test_export_traces_mask_tif()
+    # test_export_traces_within_ROI()
+    test_export_traces_chunks_only()

tests/flags_search_test.py

@@ -0,0 +1,27 @@
+from view.gui.application_settings import initialize_app_settings
+from view.gui.flags_search import get_flags_index, query
+from view.python_core.flags import FlagsManager
+import pprint
+
+
+def index_creation_test():
+    """
+    Testing creation of flags index
+    """
+
+    initialize_app_settings()
+    flags = FlagsManager()
+
+    ix = get_flags_index(flags)
+    return ix
+
+
+def query_tests():
+    """
+    Testing querying index
+    """
+    ix = index_creation_test()
+    pprint.pprint(query(index=ix, query_str="movie"))
+    pprint.pprint(query(index=ix, query_str="color"))
+    pprint.pprint(query(index=ix, query_str="mv"))
+    pprint.pprint(query(index=ix, query_str="threshold"))

tests/flags_test.py

@@ -0,0 +1,139 @@
+from view.python_core.flags import FlagsManager
+from view.python_core.utils.colors import interpret_flag_SO_MV_colortable
+from common import get_example_data_root_path
+from view.idl_translation_core.IDL import createPalette
+import numpy as np
+from matplotlib.colors import Colormap
+import os
+import tempfile
+import pathlib as pl
+
+
+def test_flags_internal():
+    """
+    Testing the initialization of flags with default values from definition csv
+    """
+
+    flags = FlagsManager()
+
+
+def test_SO_MV_SO_MV_colortable_flags():
+    """
+    Testing setting SO_MV_SO_MV_colortable values
+    """
+
+    flags = FlagsManager()
+
+    values = [11, 12, 13, 14, "jet", "autumn"]
+
+    for value in values:
+        flags.update_flags({"SO_MV_SO_MV_colortable": value})
+
+
+def test_flag_fails():
+    """
+    Testing cases for which  flag initialization must fail
+    """
+
+    flags = FlagsManager()
+
+    must_fail = [["SO_MV_SO_MV_colortable", True],  # expected value is int or str, bool given
+                 ["VIEW_VIEW_batchmode", "whatever"],  # expected value is bool, string given
+                 ["VIEW_VIEW_batchmode", 15],  # expected value is bool, incompatible integer given
+                 ["LE_loadExp", "whatever"]  # expected value is integer, incompatible string given
+                 ]
+
+    for k, v in must_fail:
+        try:
+            flags.update_flags({k: v})
+        except AssertionError as ase:
+            pass
+
+
+def test_flags_read_write():
+    """
+    Testing flags reading and writing functionalities using internal flags in FakeData test dataset
+    """
+
+    flags = FlagsManager()
+
+    data_root = get_example_data_root_path()
+
+    moaf_path = data_root / "FakeData"
+
+    flags.update_flags({"STG_MotherOfAllFolders": str(moaf_path)})
+
+    STG_flags = {"STG_OdorReportPath": "IDLoutput",
+                 "STG_OdorInfoPath": "Lists",
+                 "STG_OdormaskPath": "Coor",
+                 "STG_Datapath": "data",
+                 "STG_ProcessedDataPath": "ProcessedData",
+                 "STG_OdorAreaPath": "Areas"
+                 }
+
+    flags.update_flags(STG_flags)
+
+    flags_temp = FlagsManager()
+    flags_temp.clear_flags()
+
+    temp_yml_path = moaf_path / f"{tempfile.gettempprefix()}.yml"
+    temp_yml_filename = str(temp_yml_path)
+    flags.write_flags_to_yml(temp_yml_filename)
+    flags_temp.read_flags_from_yml(temp_yml_filename)
+
+    for flag_name in flags_temp.compound_path_flags:
+        if flag_name in flags_temp.flags:
+            path = pl.Path(flags_temp[flag_name])
+            if path.is_absolute():
+                flags_temp.update_flags({flag_name: os.path.relpath(path, flags_temp["STG_MotherOfAllFolders"])})
+
+    assert len(flags_temp.flags) == len(flags.flags)
+    assert len(set(flags_temp.flags.keys()) - set(flags.flags.keys())) == 0
+    assert all(flags.flags[x] == flags_temp.flags[x] for x in flags.flags.keys())
+    temp_yml_path.unlink()
+
+
+def test_interpret_flag_SO_MV_SO_MV_colortable():
+    """
+    Testing view.python_core.flags.interpret_flag_SO_MV_SO_MV_colortable
+    """
+
+    valid_SO_MV_SO_MV_colortable_values = [11, 12, 13, 14, "winter", "cool", "jet"]
+
+    invalid_SO_MV_SO_MV_colortable_value = [50, 150, "whatever", "notacolormap"]
+
+    for value in valid_SO_MV_SO_MV_colortable_values:
+
+        cmap, bg, fg = interpret_flag_SO_MV_colortable(value)
+
+        assert issubclass(cmap.__class__, Colormap)
+        assert len(bg) in (3, 4)
+        assert len(fg) in (3, 4)
+
+        if type(value) is int:
+
+            original_cmap = createPalette(value)
+
+            original_cols = original_cmap(np.linspace(0, 1, original_cmap.N))
+
+            interpreted_cols = cmap(np.linspace(0, 1, original_cmap.N - 2))
+
+            assert np.allclose(original_cols[1:-1, :], interpreted_cols)
+
+    for value in invalid_SO_MV_SO_MV_colortable_value:
+
+        it_failed = False
+        try:
+            interpret_flag_SO_MV_colortable(value)
+        except ValueError as ve:
+            it_failed = True
+        except NotImplementedError as se:
+            it_failed = True
+
+        assert it_failed
+
+
+if __name__ == "__main__":
+    # test_flag_fails()
+    test_flags_read_write()
+

tests/generate_overviews_tests.py

@@ -0,0 +1,267 @@
+from common import initialize_test_yml_list_measurement
+from view import VIEW
+import tifffile
+import pathlib as pl
+from nose.tools import raises
+
+
+class OverviewsGenerator(object):
+
+    def __init__(self):
+        self.test_yml, self.test_animal, self.test_measu = initialize_test_yml_list_measurement()
+
+        self.view = VIEW()
+
+        self.view.update_flags_from_ymlfile(self.test_yml)
+        self.view.update_flags(get_default_overview_flags_hack())
+
+    def generate(self, flags_to_update, suffix):
+
+        self.view.update_flags(flags_to_update)
+        self.view.load_measurement_data(self.test_animal, self.test_measu)
+        self.view.calculate_signals()
+        frame_data2write, data_limits = self.view.generate_overview_for_output_for_current_measurement()
+
+        test_folder = pl.Path(self.view.flags["STG_OdorReportPath"]) / "test_single_overviews"
+
+        if not test_folder.is_dir():
+            test_folder.mkdir()
+
+        op_file_name = test_folder / f"{self.view.flags['STG_ReportTag']}_{self.view.p1.metadata.ex_name}{suffix}.tif"
+
+        tifffile.imsave(op_file_name, data=frame_data2write, photometric="rgb")
+
+
+def get_default_overview_flags_hack():
+
+    return {"SO_Method": 0, "CTV_Method": 22, "CTV_firstframe": 25, "CTV_lastframe": 35}
+
+
+def generate_overviews(flags_to_update, suffix):
+
+    og = OverviewsGenerator()
+    og.generate(flags_to_update, suffix)
+
+
+def test_defaults():
+
+    generate_overviews(flags_to_update={}, suffix="_defaults")
+
+
+def test_no_colorbar():
+
+    generate_overviews(flags_to_update={"CTV_scalebar": 0}, suffix="_no_scalebar")
+
+
+def test_rotate_flags():
+    """
+    Testing generating overview with different rotate flags
+    """
+
+    flags = {"SO_individualScale": 3, "SO_indiScale3factor": 0.25}
+
+    for rot in range(1, 8):
+
+        flags["SO_rotateImage"] = rot
+        generate_overviews(flags, f"SO_rotateImage_{rot}")
+
+    flags["SO_reverseIt"] = True
+    generate_overviews(flags, "SO_reverseIt_True")
+
+
+def test_scale_flags():
+    """
+    Testing generating overviews with different scale flags
+    """
+
+    percentile_value = 20
+    flags_with_percentile = {"SO_percentileScale": True, "SO_percentileValue": percentile_value}
+    flags_without_percentile = {}
+
+    cutborder = 5
+    flags_with_cutborder = {"SO_cutborder": cutborder}
+
+    flag_types = {f"_percentileValue{percentile_value}": flags_with_percentile,
+                  "": flags_without_percentile,
+                  f"_cutborder{cutborder}": flags_with_cutborder}
+
+    for label, flags_to_copy in flag_types.items():
+        for indiscale in [1, 2, 4, 5, 6,
+                          11, 12, 14, 15, 16,
+                          21, 22, 24, 25, 26]:
+            flags = flags_to_copy.copy()
+            flags["SO_individualScale"] = indiscale
+            generate_overviews(flags, f"SO_individualScale{indiscale}{label}")
+
+        for indiscale in [3, 13, 23]:
+            flags = flags_to_copy.copy()
+            flags["SO_individualScale"] = indiscale
+            flags["SO_indiScale3factor"] = 0.25
+            generate_overviews(flags, f"SO_individualScale{indiscale}_factor0p25{label}")
+
+            flags = flags_to_copy.copy()
+            flags["SO_individualScale"] = indiscale
+            flags["SO_indiScale3factor"] = 0.4
+            generate_overviews(flags, f"SO_individualScale{indiscale}_factor0p4{label}")
+
+            flags = flags_to_copy.copy()
+            flags["SO_individualScale"] = indiscale
+            flags["SO_indiScale3factor"] = 0
+            generate_overviews(flags, f"SO_individualScale{indiscale}_factor0{label}")
+
+@raises(ValueError)
+def test_large_bordercut():
+    """
+    Testing generation of overview when SO_cutborder is inappropriately large
+    """
+
+    generate_overviews({"SO_cutborder": 106}, "_impossible")
+
+
+
+def test_filters():
+    """
+    Testing generating overviews with spatial filters
+    """
+
+    generate_overviews({"Signal_Signal_FilterSpaceFlag": True, "Signal_Signal_FilterSpaceSize": 3, "SO_individualScale": 3,
+                        "SO_indiScale3factor": 0.25}, "space_filter_3")
+
+    generate_overviews({"Signal_Signal_FilterSpaceFlag": True, "Signal_Signal_FilterSpaceSize": 3, "SO_withinArea": True},
+                       "space_filter_3_withinArea_true")
+
+
+def test_SO_withinArea():
+    """
+    Testing generating overviews with SO_withinArea set
+    """
+
+    generate_overviews({'SO_withinArea': True,
+                        'SO_individualScale': 3,
+                        "SO_indiScale3factor": 0.25},
+                        "SO_within_area_True")
+
+    generate_overviews({'SO_withinArea': True,
+                        'SO_thresholdShowImage': "bgColor",
+                        'SO_individualScale': 3,
+                        "SO_indiScale3factor": 0.25},
+                       "SO_within_area_True_Image_bgColor")
+
+    generate_overviews({'SO_withinArea': True,
+                        'SO_individualScale': 3,
+                        "SO_indiScale3factor": 0.25,
+                        "SO_cutborder": 5},
+                        "SO_within_area_True_cutBorder5")
+
+
+def test_thresholdOn():
+    """
+    Testing generating overviews with different values of SO_thresholdOn
+    """
+
+    for within_area in (True, False):
+
+        threshold_on_vals = {"foto1": [("a1000", "a400"), ("r50", "r30")],
+                             "overview": [("a0.05","a-0.05"), ("r50", "r30")]}
+        for threshold_on, threshold_vals in threshold_on_vals.items():
+            for (threshold_pos, threshold_neg) in threshold_vals:
+                generate_overviews({'SO_withinArea': within_area,
+                                    'SO_thresholdOn': threshold_on,
+                                    'SO_lowerThreshPositiveResps': threshold_pos,
+                                    'SO_upperThreshNegativeResps': threshold_neg,
+                                    'SO_individualScale': 3,
+                                    "SO_indiScale3factor": 0.25},
+                                    f"SO_thresholdOn_{threshold_on}"
+                                    f"_vals_{threshold_pos}{threshold_neg}"
+                                    f"_withinArea_{within_area}")
+
+
+def test_thresholdShowImage():
+    """
+    Testing generating overviews with different settings for SO_thresholdShowImage and SO_threshold_scale
+    """
+
+    for threshold_show_image in ["foto1", "bgColor"]:
+        for threshold_scale in ["full", "onlyShown"]:
+
+            generate_overviews({"SO_thresholdShowImage": threshold_show_image,
+                                "SO_thresholdScale": threshold_scale,
+                                'SO_thresholdOn': "foto1",
+                                'SO_thresholdOnValue': -1000,
+                                'SO_individualScale': 3,
+                                "SO_indiScale3factor": 0.25
+                                },
+                                f"SO_thresholdOn_foto1_val_-1000_Image_"
+                                f"{threshold_show_image}_scale_{threshold_scale}")
+
+
+def test_bgColor():
+    """
+    Testing generating overviews with SO_bgColor set
+    """
+
+    generate_overviews({'SO_withinArea': True,
+                        'SO_thresholdShowImage': "bgColor",
+                        'SO_individualScale': 3,
+                        "SO_bgColor": 'g',
+                        "SO_indiScale3factor": 0.25},
+                       "SO_within_area_True_Image_bgColor_green")
+
+def test_bgColor_fgColor():
+    """
+    Testing generating overviews with SO_bgColor set
+    """
+
+    generate_overviews({
+                        "SO_bgColor": 'y',
+                        "SO_fgColor": 'r'
+                        },
+                       "_bgColor_yellow_fg_color_red")
+
+
+def test_scale_legend_factor():
+    """
+    Testing setting SO_scaleLegendFactor
+    """
+
+    for factor in (10, 100):
+        generate_overviews({"SO_individualScale": 2,
+                            "SO_percentileScale": True,
+                            "SO_percentileValue": 20,
+                            "SO_scaleLegendFactor": factor},
+                           f"_scaleLegendFactor{factor}")
+
+
+def test_showROIs():
+    """
+    Testing overview generation with different types of perimeters
+    """
+
+    test_values = [10, 13, 14, 15, 20, 23, 24, 25]
+
+    for test_value in test_values:
+        print(f"Testing with SO_showROIs={test_value}")
+        og = OverviewsGenerator()
+
+        og.generate(flags_to_update={"SO_showROIs": test_value}, suffix=f"_showROI{test_value}")
+
+    og = OverviewsGenerator()
+    og.generate({"SO_showROIs": 15, "SO_cutborder": 5}, "showROIs15_cutborder5")
+
+
+# def test_fgColor():
+#     """
+#     Testing generating movies with SO_fgColor set
+#     """
+#
+#     generate_overviews({"SO_fgColor": "m"},
+#                         "SO_with_fgColor_magenta")
+
+
+if __name__ == '__main__':
+    # test_defaults()
+    # test_thresholdOn()
+    # test_scale_flags()
+    test_showROIs()
+
+

tests/generate_tapestries_test.py

@@ -0,0 +1,63 @@
+from common import initialize_test_yml_list_measurement
+from view import create_tapestry
+from view.python_core.flags import FlagsManager
+import pathlib as pl
+import platform
+
+
+def test_non_default():
+    """
+    Generating tapestries with non-default tapestry configs
+    """
+    test_yml, test_animal, test_measu = initialize_test_yml_list_measurement()
+
+    flags_dummy = FlagsManager()
+    flags_dummy.read_flags_from_yml(test_yml)
+
+    def text_below(row):
+        return f"{row['Odour']}_{row['OConc']}_{row['Pharma']}"
+
+    progs_path = pl.Path(flags_dummy["STG_MotherOfAllFolders"]) / "IDLprogs" / "tapestry_configs"
+    for child in progs_path.iterdir():
+        if child.suffix == ".yml" and child.name != "defult.yml":
+
+            if child.name.lower().find("linux") >= 0 and platform.system() != "Linux":
+                continue
+            elif child.name.lower().find("windows") >= 0 and platform.system() != "Windows":
+                continue
+
+            create_tapestry.description = f"Generating tapestry with {child.name}"
+            yield create_tapestry, str(child), test_yml, text_below
+
+
+def run_with_yml_name(yml_name):
+
+    test_yml, test_animal, test_measu = initialize_test_yml_list_measurement()
+
+    flags_dummy = FlagsManager()
+    flags_dummy.read_flags_from_yml(test_yml)
+
+    def text_below(row):
+        return f"{row['Odour']}_{row['OConc']}_{row['Pharma']}"
+
+    tapestry_config = pl.Path(flags_dummy["STG_MotherOfAllFolders"]) / "IDLprogs" / "tapestry_configs" / f"{yml_name}.yml"
+    create_tapestry(init_yml_flags_file=test_yml, tapestry_config_file=str(tapestry_config),
+                    text_below_func=text_below)
+
+
+def test_default():
+    """
+    Generating tapestries with default tapestry configs
+    """
+
+    run_with_yml_name("default")
+
+
+if __name__ == '__main__':
+    # test_default()
+    # run_with_yml_name("different_animals")
+    # run_with_yml_name("with_movies_stack_tif")
+    # run_with_yml_name("with_movies_libx264")
+    run_with_yml_name("custom_csv_linux")
+    # run_with_yml_name("custom_csv_windows")
+    # run_with_yml_name("different_flags")

tests/importing_pro_test.py

@@ -0,0 +1,54 @@
+from view.idl_folder_translation.pro2tapestry_conf import parse_pro_file, convert_pro_to_tapestry_config
+import pathlib as pl
+import textfsm
+import pandas as pd
+import tempfile
+
+
+def test_parsing_pro():
+    """
+    Testing parsing a .pro file
+    """
+
+    test_file = pl.Path("tests") / "test_files" / "pro_tests" / "grl_A_5803a.pro"
+    pro_data_df = parse_pro_file(test_file)
+
+    expected_csv = pl.Path("tests") / "test_files" / "pro_tests" / "grl_A_5803a_expected.csv"
+
+    pro_data_temp_op = pl.Path(tempfile.gettempdir()) / "grl_A_5803_temp_out.csv"
+
+    pro_data_df.to_csv(pro_data_temp_op)
+
+    # this works for expected files saved from linux when tested on linux and windows, filecmp.cmp does not.
+    # might have something to do line terminators
+    with open(pro_data_temp_op) as fho:
+        with open(expected_csv) as fhe:
+            assert fho.read() == fhe.read()
+
+
+def test_converting_pro():
+    """
+    Testing converting a .pro file
+    :return:
+    """
+
+    test_file = pl.Path("tests") / "test_files" / "pro_tests" / "grl_A_5803a.pro"
+
+    test_op_yml = pl.Path(tempfile.gettempdir()) / "grl_A_5803_temp_out.yml"
+
+    convert_pro_to_tapestry_config(
+        output_yml_file=test_op_yml, input_pro_file=test_file, animal_tag="A_5803a", flags_to_override={})
+
+    expected_yml = pl.Path("tests") / "test_files" / "pro_tests" / "grl_A_5803a_expected.yml"
+
+    # this works for expected files saved from linux when tested on linux and windows, filecmp.cmp does not.
+    # might have something to do line terminators
+    with open(expected_yml) as fho:
+        with open(test_op_yml) as fhe:
+            assert fho.read() == fhe.read()
+
+
+if __name__ == '__main__':
+
+    # test_parsing_pro()
+    test_converting_pro()

tests/list_name_detection_test.py

@@ -0,0 +1,80 @@
+from view.python_core.flags import FlagsManager
+from common import get_example_data_root_path
+
+
+def check_list_name_detection(yml_name, animal, expected_list_name):
+
+    flags = FlagsManager()
+    flags.read_flags_from_yml(yml_name)
+    flags.update_flags({"STG_ReportTag": animal})
+
+    assert expected_list_name == flags.get_existing_lst_file()
+
+
+def test_FID_setting_only():
+    """
+    Testing finding existing list file for the case when only settings.xls files are present
+    :return:
+    """
+
+    data_root = get_example_data_root_path()
+
+    project_root = data_root / "LM_Till_only_FID"
+    test_yml = str(project_root/ "usage_till.yml")
+    animal = "LM_GC-FID_or22a_170816a"
+    expected_list = str(project_root / "lists" / "LM_GC-FID_or22a_170816a.settings.xls")
+
+    check_list_name_detection(yml_name=test_yml, animal=animal, expected_list_name=expected_list)
+
+
+def test_FID_LSTMixed():
+    """
+    Testing finding existing list file for the case when lst.xls and .lst files are present
+    :return:
+    """
+
+    data_root = get_example_data_root_path()
+
+    project_root = data_root / "HS_Till"
+    test_yml = str(project_root / "usage_till.yml")
+    animal = "HS_bee_PELM_180416b"
+    expected_list = str(project_root / "IDLlist" / "HS_bee_PELM_180416b.lst.xls")
+
+    check_list_name_detection(yml_name=test_yml, animal=animal, expected_list_name=expected_list)
+
+
+def test_FID_XLSMixed():
+    """
+    Testing finding existing list file for the case when lst.xls and .settings.xls files are present
+    :return:
+    """
+
+    data_root = get_example_data_root_path()
+
+    project_root = data_root / "Or47a_test"
+    test_yml = str(project_root / "usage_till_test.yml")
+    animal = "AL_190506a_or47a"
+    expected_list = str(project_root / "02_SETTINGS" / "AL_190506a_or47a.lst.xls")
+
+    check_list_name_detection(yml_name=test_yml, animal=animal, expected_list_name=expected_list)
+
+
+def test_FID_LSTXLS_only():
+    """
+    Testing finding existing list file for the case when only lst.xls files are present
+    :return:
+    """
+
+    data_root = get_example_data_root_path()
+
+    project_root = data_root / "SS_LSM"
+    test_yml = str(project_root / "usage_lsm.yml")
+    animal = "2019_08_15_locust_oregon green"
+    expected_list = str(project_root / "Lists" / "2019_08_15_locust_oregon green.lst.xls")
+
+    check_list_name_detection(yml_name=test_yml, animal=animal, expected_list_name=expected_list)
+
+
+if __name__ == '__main__':
+
+    test_FID_setting_only()

tests/stim_param_test.py

@@ -0,0 +1,77 @@
+from view.python_core.measurement_list import MeasurementList
+import pathlib as pl
+import tempfile
+
+
+def valid_stim_params_test():
+    """
+    Testing whether valid stim paramter settings do not cause an error
+    :return:
+    """
+
+    valid_stim_list = 'tests/test_files/measurement_test_files/190112_locust_ip_VALID_stims.lst.xls'
+
+    ml = MeasurementList.create_from_lst_file(valid_stim_list, LE_loadExp=4)
+    for measu_index, measu in enumerate(ml.get_measus()):
+        p1_metadata, extra_metadata = ml.get_p1_metadata_by_index(measu_index)
+
+
+def invalid_stim_params_test():
+    """
+    Testing whether invalid stim parameter settings do  cause an error
+    :return:
+    """
+
+    invalid_stim_list = 'tests/test_files/measurement_test_files/190112_locust_ip_INVALID_stims.lst.xls'
+
+    ml = MeasurementList.create_from_lst_file(invalid_stim_list, LE_loadExp=4)
+    for measu_index, measu in enumerate(ml.get_measus()):
+        try:
+            p1_metadata, extra_metadata = ml.get_p1_metadata_by_measu(measu)
+        except ValueError as e:
+            pass
+        except AssertionError as e:
+            pass
+
+
+def stim_spec_test_generator():
+
+    test_root = "tests/test_files/measurement_test_files/valid_files"
+
+    test_root_path = pl.Path(test_root)
+    dirs = [child for child in test_root_path.iterdir() if child.is_dir()]
+
+    for direc in dirs:
+        for child in direc.iterdir():
+            if child.suffix == ".xls":
+                try_importing_measurement_list.description = f"Testing with the stimulus specification in " \
+                                                             f"{child.relative_to(test_root_path)}"
+                yield try_importing_measurement_list, str(child)
+
+
+def try_importing_measurement_list(xls):
+
+    expected_csv = f"{xls.split('.')[0]}.csv"
+    ml = MeasurementList.create_from_lst_file(xls, LE_loadExp=666)
+    p1_metadata, extra_metadata = ml.get_p1_metadata_by_measu(1)
+    temp_out = str(pl.Path(tempfile.gettempdir()) / "view_temp.csv")
+    temp = p1_metadata["pulsed_stimuli_handler"].stimulus_frame.copy()
+    del temp["Sampling Period"]
+    temp.to_csv(str(temp_out))
+
+    # this works for expected files saved from linux when tested on linux and windows, filecmp.cmp does not.
+    # might have something to do line terminators
+    with open(temp_out) as fho:
+        with open(expected_csv) as fhe:
+            assert fho.read() == fhe.read()
+
+
+if __name__ == "__main__":
+
+    # valid_stim_params_test()
+    try_importing_measurement_list("tests/test_files/measurement_test_files/valid_files/"
+                                   "one_stim/stimON_stimOFF.lst.xls")
+    # try_importing_measurement_list("tests/test_files/measurement_test_files/valid_files/"
+    #                                "two_stim_new_mixed/stimON_stimOFF_stimLen.lst.xls")
+
+

tests/test_artifact_correction.py

@@ -0,0 +1,185 @@
+from view import VIEW
+from view.python_core.io import write_tif_2Dor3D
+from common import initialize_test_yml_list_measurement
+
+
+def run_artifact_correction(flags_to_update, output_suffix=None):
+
+    test_yml, test_animal, test_measu = initialize_test_yml_list_measurement()
+
+    flags = {
+        "LE_BleachCorrMethod": "None",
+        "LE_BleachExcludeStimulus": False,
+        "LE_BleachExcludeArea": False,
+        "LE_BleachCutBorder": 0,
+        "LE_ScatteredLightFactor": 0,
+        "LE_ScatteredLightRadius": 50
+    }
+
+    flags.update(flags_to_update)
+
+    vo = VIEW()
+    vo.update_flags_from_ymlfile(yml_filename=test_yml)
+    if isinstance(flags, dict):
+        vo.update_flags(flags)
+
+    vo.initialize_animal(animal=test_animal)
+    vo.load_measurement_data_from_current_animal(measu=test_measu)
+
+    if output_suffix is not None:
+        op_dir = vo.flags.get_processed_data_dir_path() / "Artifact_corrected_raw"
+        op_dir.mkdir(exist_ok=True)
+        op_filename = op_dir / f"{test_animal}_{test_animal}{output_suffix}.tif"
+        write_tif_2Dor3D(array_xy_or_xyt=vo.p1.raw1, tif_file=op_filename)
+
+
+def test_no_bleach_method():
+    """
+    testing loading data without bleach correction
+    """
+
+    run_artifact_correction(
+        flags_to_update={},
+        # output_suffix="_BC_None"
+    )
+
+
+def test_no_bleach_with_scatter_light_correction():
+    """
+    testing loading data without bleach correction, but with scatter light correction
+    """
+
+    run_artifact_correction(
+        flags_to_update={"LE_ScatteredLightFactor": 1},
+        # output_suffix="_BC_None"
+    )
+
+
+def test_log_bleach_pixelwise():
+    """
+    testing loading data using pixelwise log bleach correction
+    """
+
+    run_artifact_correction(
+        flags_to_update={
+            "LE_BleachCorrMethod": "log_pixelwise"
+        },
+        # output_suffix="_BC_log_pixelwise"
+    )
+
+
+def test_log_bleach_pixelwise_excluding_area():
+    """
+    testing loading data using pixelwise log bleach correction with area exclusion
+    """
+
+    run_artifact_correction(
+        flags_to_update={
+            "LE_BleachCorrMethod": "log_pixelwise",
+            "LE_BleachExcludeArea": True
+        },
+        # output_suffix="_BC_log_pixelwise_excludingArea"
+    )
+
+
+def test_log_bleach_pixelwise_excluding_stimulus():
+    """
+    testing loading data using pixelwise log bleach correction with stimulus exclusion
+    """
+
+    run_artifact_correction(
+        flags_to_update={
+            "LE_BleachCorrMethod": "log_pixelwise",
+            "LE_BleachExcludeStimulus": True,
+            "LELog_ExcludeSeconds": 5,
+            "LE_PrestimEndBackground": 5
+        },
+        # output_suffix="_BC_log_pixelwise_excludingStimulus"
+    )
+
+
+def test_log_bleach_uniform():
+    """
+    testing loading data using uniform log bleach correction
+    """
+
+    run_artifact_correction(
+        flags_to_update={
+            "LE_BleachCorrMethod": "log_uniform"
+        },
+        # output_suffix="_BC_log_uniform"
+    )
+
+
+def test_log_bleach_uniform_excluding_area():
+    """
+    testing loading data using uniform log bleach correction with area exclusion
+    """
+
+    run_artifact_correction(
+        flags_to_update={
+            "LE_BleachCorrMethod": "log_uniform",
+            "LE_BleachExcludeArea": True
+        },
+        # output_suffix="_BC_log_uniform_excludingArea"
+    )
+
+
+def test_log_bleach_uniform_excluding_stimulus():
+    """
+    testing loading data using uniform log bleach correction with stimulus exclusion
+    """
+
+    run_artifact_correction(
+        flags_to_update={
+            "LE_BleachCorrMethod": "log_uniform",
+            "LE_BleachExcludeStimulus": True,
+            "LELog_ExcludeSeconds": 5,
+            "LE_PrestimEndBackground": 5
+        },
+        # output_suffix="_BC_log_uniform_excludingStimulus"
+    )
+
+
+def test_artifact_correction_filters_only():
+    """
+    testing loading data with no bleach correction, but with median filtering
+    """
+
+    flags_to_test = [
+        {"Data_Median_Filter": 0},
+        {"Data_Median_Filter": 1},
+        {"Data_Median_Filter": 2},
+        {"Data_Median_Filter": 3, "Data_Median_Filter_space": 2, "Data_Median_Filter_time": 3},
+        {"Data_Mean_Filter": 0},
+        {"Data_Mean_Filter": 1},
+        {"Data_Mean_Filter": 2},
+        {"Data_Mean_Filter": 3, "Data_Mean_Filter_space": 10, "Data_Mean_Filter_time": 10}
+    ]
+
+    for flags in flags_to_test:
+
+        run_artifact_correction.description = f"Testing raw data filtering with {flags}"
+        yield run_artifact_correction, flags
+
+
+if __name__ == '__main__':
+
+    # test_no_bleach_method()
+    # test_log_bleach_pixelwise()
+    # test_log_bleach_pixelwise_excluding_stimulus()
+    test_log_bleach_uniform()
+    # test_log_bleach_uniform_excluding_area()
+    # test_log_bleach_uniform_excluding_stimulus()
+
+    # test_no_bleach_with_scatter_light_correction()
+
+    # run_artifact_correction(
+    #     flags_to_update={"Data_Median_Filter": 3, "Data_Median_Filter_space": 10, "Data_Median_Filter_time": 10},
+    #     output_suffix="_MedianFilteredSpace10Time10"
+    # )
+    #
+    # run_artifact_correction(
+    #     flags_to_update={"Data_Mean_Filter": 3, "Data_Mean_Filter_space": 10, "Data_Mean_Filter_time": 10},
+    #     output_suffix="_MeanFilteredSpace10Time10"
+    # )

tests/test_ctvs.py

@@ -0,0 +1,60 @@
+from common import initialize_test_yml_list_measurement
+from view import VIEW
+from view.python_core import ctvs
+from view.python_core.overviews import generate_overview_frame
+import inspect
+import pathlib as pl
+import numpy as np
+
+
+def ctv_signatures_test():
+    """
+    Check if signatures of all csv functions are equal
+    """
+    ctv_funcs = [x for x in inspect.getmembers(ctvs) if callable(x)]
+
+    for ctv_func in ctv_funcs[1:]:
+
+        assert inspect.signature(ctv_func) == inspect.signature(ctv_funcs[0])
+
+
+def check_ctv_generic(ctv_method):
+
+    test_yml, test_animal, test_measu = initialize_test_yml_list_measurement()
+
+    view = VIEW()
+
+    view.update_flags_from_ymlfile(test_yml)
+
+    view.update_flags({"CTV_Method": ctv_method, "SO_Method": 0})
+    view.load_measurement_data(test_animal, test_measu)
+    view.calculate_signals()
+
+    overview_frames = generate_overview_frame(flags=view.flags, p1=view.p1)
+    overview = overview_frames[0, :, :]
+
+    expected_overview_fle_path = \
+        pl.Path(view.flags["STG_MotherOfAllFolders"]) / "test_files" / f"ctv{ctv_method}_expected.npz"
+
+    expected_overview = np.load(str(expected_overview_fle_path))["expected_overview"]
+    assert np.allclose(overview, expected_overview)
+
+
+def test_ctv_22():
+    """
+    Testing CTV 22 with FakeData
+    """
+
+    check_ctv_generic(22)
+
+
+def test_ctv_35():
+    """
+    Testing CTV 35 with FakeData
+    """
+
+    check_ctv_generic(35)
+
+
+if __name__ == '__main__':
+    test_ctv_35()

tests/test_files/measurement_test_files/valid_files/one_stim/stimON_stimLen.csv

@@ -0,0 +1,2 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:06

tests/test_files/measurement_test_files/valid_files/one_stim/stimON_stimOFF.csv

@@ -0,0 +1,2 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000

tests/test_files/measurement_test_files/valid_files/one_stim/stimON_stimOFF_stimLen.csv

@@ -0,0 +1,2 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000

tests/test_files/measurement_test_files/valid_files/one_stim/stimONms_stimLen.csv

@@ -0,0 +1,2 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:06

tests/test_files/measurement_test_files/valid_files/one_stim/stimONms_stimOFF.csv

@@ -0,0 +1,2 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000

tests/test_files/measurement_test_files/valid_files/two_stim_new/stimON_stimLen.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:06
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:11

tests/test_files/measurement_test_files/valid_files/two_stim_new/stimON_stimOFF.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:12.200000

tests/test_files/measurement_test_files/valid_files/two_stim_new/stimON_stimOFF_stimLen.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:12.200000

tests/test_files/measurement_test_files/valid_files/two_stim_new/stimONms_stimLen.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:06
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:11

tests/test_files/measurement_test_files/valid_files/two_stim_new/stimONms_stimOFF.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:12.200000

tests/test_files/measurement_test_files/valid_files/two_stim_new_mixed/stimON_stimOFF_stimLen.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:11

tests/test_files/measurement_test_files/valid_files/two_stim_old/stimON_stimLen.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:06
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:11

tests/test_files/measurement_test_files/valid_files/two_stim_old/stimON_stimOFF.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:12.200000

tests/test_files/measurement_test_files/valid_files/two_stim_old/stimON_stimOFF_stimLen.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:12.200000

tests/test_files/measurement_test_files/valid_files/two_stim_old/stimONms_stimLen.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:06
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:11

tests/test_files/measurement_test_files/valid_files/two_stim_old/stimONms_stimOFF.csv

@@ -0,0 +1,3 @@
+,Odor,Concentration,Pulse Start Time,Pulse End Time
+0,FAKE,-10.0,0 days 00:00:05,0 days 00:00:07.200000
+1,FAKE,-10.0,0 days 00:00:10,0 days 00:00:12.200000

tests/test_files/pro_tests/grl_A_5803a.pro

@@ -0,0 +1,48 @@
+
+pro grl_A_5803a
+common data
+common CFD
+common CFDconst
+common Vars
+
+;butanol
+;scalemax =
+;scalemin =
+
+;overall
+scalemax = 23.1
+scalemin = -3.0
+
+
+flag[rm_newcolumn]=1
+subloop,'3';  Long00
+flag[rm_newcolumn]=0
+subloop,'5';  Long01
+
+flag[rm_newcolumn]=1
+subloop,'7';  Long02
+flag[rm_newcolumn]=0
+subloop,'9';  Long03
+subloop,'10';  Long04
+subloop,'11';  Long05
+subloop,'12';  Long06
+subloop,'13';  Long07
+subloop,'14';  Long08
+subloop,'15';  Long09
+
+
+flag[rm_newcolumn]=1
+subloop,'19';  Long10
+flag[rm_newcolumn]=0
+subloop,'21';  Long11
+subloop,'22';  Long12
+subloop,'23';  Long13
+subloop,'24';  Long14
+subloop,'25';  Long15
+subloop,'26';  Long16
+subloop,'27';  Long17
+
+
+end
+
+

tests/test_files/pro_tests/grl_A_5803a_expected.csv

@@ -0,0 +1,5 @@
+,flag_names,flag_values,measus
+0,"['scalemax', 'scalemin']","['23.1', '-3.0']",[]
+1,[],[],"['3', '5']"
+2,[],[],"['7', '9', '10', '11', '12', '13', '14', '15']"
+3,[],[],"['19', '21', '22', '23', '24', '25', '26', '27']"

tests/test_files/pro_tests/grl_A_5803a_expected.yml

@@ -0,0 +1,29 @@
+row1:
+  animal: A_5803a
+  flags:
+    SO_MV_scalemax: '23.1'
+    SO_MV_scalemin: '-3.0'
+    SO_individualScale: 0
+  measus:
+  - 3
+  - 5
+row2:
+  measus:
+  - 7
+  - 9
+  - 10
+  - 11
+  - 12
+  - 13
+  - 14
+  - 15
+row3:
+  measus:
+  - 19
+  - 21
+  - 22
+  - 23
+  - 24
+  - 25
+  - 26
+  - 27

tests/test_files/valid_flag_files/view_flags_all_defaults.yml

@@ -0,0 +1,90 @@
+CSM_Datashift: 0
+CSM_Median: 0
+CSM_Median_space: 3
+CSM_Median_time: 3
+CSM_Movement: 0
+CSM_SkipFrmAtBack: 0
+CSM_SkipFrmUpFront: 0
+CTVM_Method: 0
+CTV_Method: 22
+CTV_firstframe: 22
+CTV_lastframe: 36
+CTV_scalebar: false
+CorrectFlag: false
+FT_AllOdors: false
+RM_Radius: 5
+FT_SelectFrame: false
+FT_Subset: false
+FT_TimeXAxis: false
+FilterSpaceFlag: false
+FilterSpaceSize: 3
+FilterTimeFlag: false
+FilterTimeSize: 3
+LE_AskForAir: false
+LE_BleachStartFrame: 2
+LE_ClipPixels: 0
+LE_FirstBuffer: 1
+LE_LogExcludeSeconds: 0
+LE_LogInitialFactor: 1
+LE_PrestimEndBackground: 2
+LE_ShrinkFaktor: 0
+LE_StartBackground: 4
+MotherOfAllFolders: not set yet
+PTA_PlotMeanValue: false
+PTA_PlotTimeRange: false
+RM_FotoOk: 1
+RM_NewColumn: false
+RM_NextPosition: (0, 0)
+RM_PlotTrace: false
+RM_PrintAscii: false
+RM_PrintLine: false
+RM_ROITrace: false
+RM_differentViews: false
+RM_separateLayers: 0
+RM_unsharpmask: false
+SO_Method: 10
+SO_individualScale 0
+SO_withinArea: 0
+STG_Datapath: not set yet
+STG_Measu: -1
+STG_OdorInfoPath: not set yet
+STG_OdorReportPath: not set yet
+STG_OdormaskPath: not set yet
+STG_ReportTag: not set yet
+VIEW_DeleteRawData: 1
+VIEW_InitCorr: 0
+VIEW_MultiExp: 0
+VIEW_No4Darray: 0
+VIEW_ReportMethod: 10
+VIEW_ScatterLightFactor: 0.0
+batchmode: false
+colortable: 13
+mv_FirstFrame: 0
+mv_LastFrame: -1
+mv_SpeedFactor: 1.0
+mv_bgColor: k
+mv_bitrate: 1024k
+mv_correctStimulusOnset: 0
+mv_cutborder: 0.0
+mv_displayTime: true
+mv_exportFormat: libx264
+mv_fgColor: w
+mv_indiScale3factor: 0.2
+mv_individualScale: 0
+mv_markStimulus: 0
+mv_minimumBrightness: 0.0
+mv_morphoThreshold: 0.0
+mv_percentileScale: false
+mv_percentileValue: 0.0
+mv_realTime: 0
+mv_reverseIt: false
+mv_rotateImage: 0
+mv_sdSignificanceCut: 0.0
+mv_suppressMilliseconds: false
+mv_withinArea: false
+mv_xgap: 0
+mv_ygap: 0
+scalemax: 1.0
+scalemin: 0.0
+view_CalcMethod: 3900
+view_loadExp: 3

tests/test_io.py

@@ -0,0 +1,47 @@
+from common import initialize_test_yml_list_measurement
+from view import VIEW
+from view.python_core.io import read_tif_2Dor3D, write_tif_2Dor3D
+import tempfile
+import pathlib as pl
+import numpy as np
+
+
+def test_tif_io():
+    test_yml, test_animal, test_measu = initialize_test_yml_list_measurement()
+
+    view = VIEW()
+
+    view.update_flags_from_ymlfile(test_yml)
+
+    view.initialize_animal(test_animal)
+    view.load_measurement_data_from_current_animal(test_measu)
+
+    temp_tif_fn = str(pl.Path(tempfile.gettempdir()) / f"{tempfile.gettempprefix()}.tif")
+
+    write_tif_2Dor3D(view.p1.raw1, temp_tif_fn)
+
+    read_stack, labels = read_tif_2Dor3D(temp_tif_fn)
+
+    assert np.allclose(view.p1.raw1, read_stack)
+    assert labels is None
+
+    write_tif_2Dor3D(view.p1.raw1[:, :, 0], temp_tif_fn, labels="test")
+
+    read_stack, labels = read_tif_2Dor3D(temp_tif_fn)
+
+    assert np.allclose(view.p1.raw1[:, :, 0], read_stack)
+    assert labels == ["test"]
+
+    fake_labels = [str(x) for x in range(view.p1.raw1.shape[2])]
+    write_tif_2Dor3D(view.p1.raw1, temp_tif_fn, labels=fake_labels)
+
+    read_stack, read_labels = read_tif_2Dor3D(temp_tif_fn)
+
+    assert np.allclose(view.p1.raw1, read_stack)
+    assert all(x == y for x, y in zip(fake_labels, read_labels))
+
+
+if __name__ == '__main__':
+
+    test_tif_io()
+

tests/test_loading_data.py

@@ -0,0 +1,99 @@
+from common import get_example_data_root_path, initialize_test_yml_list_measurement
+
+from view.python_core.flags import FlagsManager
+from view.python_core.paths import get_existing_raw_data_filename
+from view.python_core.view_object import VIEW
+from view.python_core.p1_class import get_empty_p1
+
+
+def load_calc_data(yml_file, animal, measus=None, flags=None):
+
+    vo = VIEW()
+    vo.update_flags_from_ymlfile(yml_filename=yml_file)
+    if isinstance(flags, dict):
+        vo.update_flags(flags)
+
+    vo.initialize_animal(animal=animal)
+
+    for measu in vo.get_measus_for_current_animal(analyze_values_to_use=(1, 2)):
+        if measus is None or measu in measus:
+            print(f"Doing animal={animal}, measu={measu}")
+            vo.load_measurement_data_from_current_animal(measu=measu)
+            vo.calculate_signals()
+
+    return vo
+
+
+def test_loading_all_data():
+
+    yml_animal_dict = {
+        "FakeData/test_defaults.yml": ["FakeData"],
+        "HS_Till/usage_till.yml": ["HS_bee_PELM_180416b", "HS_bee_PELM_180424b"],
+        "IP_Fura/usage_till.yml": ["190112_locust_ip", "190112_locust_ip2", "190529_locust_ip31"],
+        "LM_Till_only_FID/usage_till.yml":
+            ["LM_GC-FID_or22a_170816a", "LM_GC-FID_or22a_170816b", "LM_GC-FID_or22a_170816c"],
+        "MR_Till/usage_till.yml": ["MR_190613c_or47a", "MR_190614a_or47a"],
+        "MS_LSM/usage_lsm.yml": ["2020_02_06_OK107_GCaMP6f", "testview"],
+        "Or47a_test/usage_till_test.yml":
+            ["AL_190506a_or47a", "MR_190510b_or47a", "MR_190515b_or47a", "PG_190702a_or47a"],
+        "SS_LSM/usage_lsm.yml": ["2019_08_15_locust_oregon green"]
+    }
+
+    example_data_root_path = get_example_data_root_path()
+
+    for yml_relative_path, animals in yml_animal_dict.items():
+
+        yml_file = str(example_data_root_path / yml_relative_path)
+
+        for animal in animals:
+
+            load_calc_data.description \
+                = f"Testing loading data and signal calculation with yml={yml_relative_path} and animal={animal}"
+
+            yield load_calc_data, yml_file, animal
+
+
+def test_loading_data_without_measurement_list():
+    """Testing loading data without measurement list"""
+    example_data_root_path = get_example_data_root_path()
+    yml_file = example_data_root_path / "HS_Till" / "usage_till.yml"
+    animal = "HS_bee_PELM_180416b"
+
+    flags = FlagsManager()
+    flags.read_flags_from_yml(yml_file)
+
+    view_obj = VIEW(flags=flags)
+
+    raw_data_file \
+        = example_data_root_path / "HS_Till" / "data" / "HS_bee_PELM_180416b.pst" / "dbb12DF.pst"
+
+    view_obj.load_measurement_data_without_list_file(
+        LE_loadExp=3,
+        raw_data_files=[raw_data_file],
+        sampling_rate=1 / 0.6,
+        animal=animal)
+
+
+if __name__ == '__main__':
+
+    # load_calc_data(
+    #     yml_file="/home/ajay/SharedWithWindows/view_test_data/HS_Till/usage_till.yml",
+    #     animal="HS_bee_PELM_180416b")
+
+    # load_calc_data(
+    #     yml_file="/home/ajay/SharedWithWindows/view_test_data/FakeData/test_defaults.yml",
+    #     animal="FakeData", flags={"LE_CalcMethod": 4})
+
+    # load_calc_data(yml_file="/mnt/data/ViewData/IP_Fura/view_flags_all_defaultsFURA.yml",
+    #                animal="190112_locust_ip2", flags={"LE_ScatteredLightFactor": 1})
+
+    # load_calc_data(
+    #     yml_file="/home/aj/SharedWithWindows/SS_LSM/usage_lsm.yml",
+    #     animal="2019_08_09_locust_calcium green")
+
+    load_calc_data(
+        yml_file="/home/aj/SharedWithWindows/pyview_test_data/Bente_Test/Bente_Test_2021.yml",
+        animal="190815_h2_El_test")
+
+    # test_loading_data_without_measurement_list()
+

tests/test_measurement_list.py

@@ -0,0 +1,122 @@
+from common import get_example_dataset_roots
+from view.python_core.measurement_list import MeasurementList
+from view.python_core.flags import FlagsManager
+import pathlib as pl
+
+
+def measurement_list_manager_loading(ext):
+
+    def exclusion_check(yml_filename):
+
+        yml_filename_lower = yml_filename.lower()
+
+        to_exclude = ["fid_hanna9", "fidor", "log2settings"]
+
+        return any([yml_filename_lower.find(x) >= 0 for x in to_exclude])
+
+    for dataset_root in get_example_dataset_roots():
+
+        yml_files = [x for x in dataset_root.iterdir() if x.suffix == ".yml" and not exclusion_check(x.name)]
+
+        if len(yml_files):
+
+            yml_file = yml_files[0]
+            flags = FlagsManager()
+
+            try:
+                flags.read_flags_from_yml(yml_file)
+
+                list_dir = pl.Path(flags["STG_OdorInfoPath"])
+
+                if list_dir.is_dir():
+                    for fle in list_dir.iterdir():
+
+                        if fle.name.endswith(ext) and not fle.name.startswith("."):
+
+                            measurement_list = MeasurementList.create_from_lst_file(str(fle), LE_loadExp=3)
+                            yield measurement_list
+            except FileNotFoundError as fnfe:
+                pass
+
+
+def test_reading_lst():
+    """
+    Testing importing lst files into view.python_core.managers.measurement_list.LSTList
+    """
+
+    for lst in measurement_list_manager_loading(".lst"):
+        pass
+
+
+def test_reading_settingsXLS():
+    """
+    Testing importing settings files into view.python_core.managers.measurement_list.SettingsXLSList
+    """
+
+    for lst in measurement_list_manager_loading(".settings.xls"):
+        pass
+
+
+def test_reading_LSTXLS():
+    """
+    Testing importing settings files into view.python_core.managers.measurement_list.LSTXLSList
+    """
+
+    for lst in measurement_list_manager_loading(".lst.xls"):
+        pass
+
+
+def run_get_p1_all(lst):
+
+    for ind, measu in enumerate(lst.get_measus()):
+
+        p1_metadata, extra_metadata = lst.get_p1_metadata_by_index(ind)
+        pass
+
+
+def test_lst2p1():
+    """
+    Testing metadata in lst files to p1
+    """
+
+    for ind, lst in enumerate(measurement_list_manager_loading(".lst")):
+        run_get_p1_all(lst)
+
+
+def test_settings2p1():
+    """
+    Testing metadata in settings files to p1
+    """
+
+    for ind, lst in enumerate(measurement_list_manager_loading(".settings.xls")):
+        run_get_p1_all(lst)
+
+
+def test_lstxls2p1():
+    """
+    Testing metadata in lst.xls files to p1
+    """
+
+    for ind, lst in enumerate(measurement_list_manager_loading(".lst.xls")):
+        run_get_p1_all(lst)
+
+
+if __name__ == "__main__":
+
+    test_reading_lst()
+
+    # print all lists in all test data sets
+    # print("Legacy LST files")
+    # for ml in measurement_list_manager_loading(".lst"):
+    #     print(ml.last_measurement_list_fle)
+    #
+    # print("LST XLS files")
+    # for ml in measurement_list_manager_loading(".lst.xls"):
+    #     print(ml.last_measurement_list_fle)
+    #
+    # print("Settings XLS files")
+    # for ml in measurement_list_manager_loading(".settings.xls"):
+    #     print(ml.last_measurement_list_fle)
+
+
+

tests/test_overview_popup_gui.py

@@ -0,0 +1,34 @@
+from common import initialize_test_yml_list_measurement
+from view import VIEW
+from view.python_core.overviews import pop_show_overview
+
+
+def test_different_configs():
+    """
+    Testing different configurations of GUI pop up window for overviews
+    """
+
+    test_yml, test_animal, test_measu = initialize_test_yml_list_measurement()
+
+    vo = VIEW()
+    vo.update_flags_from_ymlfile(test_yml)
+    vo.load_measurement_data(animal=test_animal, measu=test_measu)
+    vo.calculate_signals()
+
+    vo.update_flags({"CTV_Method": "22and35", "SO_individualScale": 3})
+
+    pop_show_overview.description = "Testing defaults"
+    yield pop_show_overview, vo.flags, vo.p1, "test", None, None
+
+    for stim_nr, feature_nr in [([0], [0]), ([0], "all"), ("all", [0]), ("all", "all")]:
+        pop_show_overview.description = f"Testing stimulus number={stim_nr} and feature number={feature_nr}"
+        yield pop_show_overview, vo.flags, vo.p1, "test", stim_nr, feature_nr
+
+
+if __name__ == '__main__':
+
+    for args in test_different_configs():
+
+        args[0](*args[1:])
+        input("Press any key to close figure and continue...")
+

utility_scripts/replace_flag_names.py

@@ -0,0 +1,52 @@
+import sys
+import pathlib as pl
+import pandas as pd
+
+
+def main(fle_or_dir, replacement_map_csv):
+
+    map_df = pd.read_csv(replacement_map_csv)
+
+    path = pl.Path(fle_or_dir)
+
+    if path.is_file():
+        files = [path]
+    elif path.is_dir():
+        files = path.rglob("*")
+    else:
+        raise FileNotFoundError(f"The specified path is neither a file nor a directory!\n{fle_or_dir}")
+
+    for file in files:
+        if file.is_file() \
+                and file != pl.Path(replacement_map_csv) \
+                and not file.name.startswith(".")\
+                and str(file).find(".git") < 0:
+
+            changes_made = False
+            with open(file, "r+") as fh:
+                try:
+                    s_read = fh.read()
+                    s = f"{s_read:s}"
+                    for ind, row in map_df.iterrows():
+                        s = s.replace(row["Old Flag Name"], row["Flag Name"])
+                        s = s.replace(row["Old Flag Subgroup"], row["Flag Subgroup"])
+                    if s != s_read:
+                        changes_made = True
+                except UnicodeDecodeError as ucde:
+                    pass  # happens if the file is not a text file encoded in unicode
+
+                if changes_made:
+                    file.unlink()
+
+                    with open(file, "w") as fh:
+                        fh.write(s)
+
+
+if __name__ == '__main__':
+
+    assert len(sys.argv) == 3, \
+        f"Could not understand command. Please use as\n" \
+        f"python {__file__} <path to file or directory> <CSV file containing replacement map>"
+
+    main(sys.argv[1], sys.argv[2])
+

utility_scripts/setup_testing.py

@@ -0,0 +1,55 @@
+from PyQt5.QtCore import QSettings
+from easygui import diropenbox, ynbox, msgbox
+import pathlib as pl
+
+from view.gui.application_settings import get_view_qsettings_manager
+
+
+def main():
+    settings = get_view_qsettings_manager()
+    existing_test_data_path_str = None
+    if settings.contains("view_test_data_path"):
+        existing_test_data_path_str = settings.value("view_test_data_path")
+        existing_test_data_path = pl.Path(existing_test_data_path_str)
+        if existing_test_data_path.is_dir() and len(list(existing_test_data_path.iterdir())):
+            ch = ynbox(
+                title="Test data found!",
+                msg=f"View has been previously configured to use the folder below for storing test data. "
+                    f"This folder exists and is not empty, hence it most likely contains the correct test data. "
+                    f"What would you like to do?"
+                    f"\n\n{existing_test_data_path_str}",
+                choices=["Use a new path and download test data again (~3.3GiB)", "Use the same path as above"],
+                default_choice="Use the same path as above",
+                cancel_choice="Use the same path as above"
+            )
+
+            if ch:
+                existing_test_data_path_str = None
+
+        else:
+            existing_test_data_path_str = None
+
+    if existing_test_data_path_str is None:
+        msgbox(
+            title="Info",
+            msg="Please choose a folder in the next dialog for storing VIEW test data. Since it is ~3.3GiB is size, "
+                "we recommend creating a new folder for it")
+        file = diropenbox(title="Please choose a folder for storing VIEW test data")
+
+        if file is None:
+            raise IOError("User Abort!")
+
+        settings.setValue("view_test_data_path", file)
+        existing_test_data_path_str = file
+
+        # raise NotImplementedError  # TODO download view test data to this folder
+
+    msgbox(
+        title="View test data path",
+        msg=f"View has been configured to use data in the following folder for testing:"
+            f"\n\n{existing_test_data_path_str}")
+
+
+if __name__ == '__main__':
+
+    main()

view/__init__.py

@@ -0,0 +1,9 @@
+from pkg_resources import get_distribution
+
+__version__ = get_distribution(__name__).version
+
+from .python_core.view_object import VIEW
+from .python_core.tapestries import create_tapestry
+
+
+

view/flags_and_metadata_definitions/glodatamix_columns_doc.csv

@@ -0,0 +1,52 @@
+Column name,current/legacy,mandatory/optional,Description,
+Control,current,optional,Measu/NNumMeasu of the control measurement,
+Cycle,current,mandatory,"inverse of frequency, in ms. Set to 0 for uneven frequencies",
+FrameSizeX,current,optional,Number of pixel in acquisition frame along X,
+FrameSizeY,current,optional,Number of pixel in acquisition frame along Y,
+NoFrames,current,mandatory,Number of frames in the measurement,
+OConc,current,optional,"concentration of odour stimulus, always TENFOLD logarithmic, i.e. -23 is 10 to the -2.3",
+PhConc,current,optional,"1 for treatment, eg 1: ptx; 0:  wash or pre-ptx, includes concentration of treatment",
+PosZ,current,optional,depth of Z-plane,
+PxSzX,current,optional,Image resolution of Ca imaging video data along the X axis,
+PxSzY,current,optional,Image resolution of Ca imaging video data along the Y axis,
+TraceOffset,current,optional,Arbitary delay of trace onset in seconds,
+StimISI,current,optional,interval between stimuli,
+UTC,current,optional,Posix time stamp of measurement start,
+Age,current,optional,age of the animal in days(exact number or a range. e.g 2-4),
+Comment,current,optional,any comment,
+DBB1,current,optional,raw data file name (of the 340nm measurement for FURA),
+DBB2,current,optional,raw data file name (of the 380nm measurement for FURA),
+DBB3,current,optional,raw data file name (when required),
+GloInfo,current,optional,comment text for the glomerulus,
+GloTag,current,mandatory,integer name of glomerulus,
+Label,current,optional,label for the measurement entered in TILL VISION,
+Lambda,current,optional,Monochromator wavelength,
+MTime,current,optional,"time since first evidence that the animal is under the microscope, formatted as 'HH:MM:SS.ssss'",
+Pharma,current,optional,text of pharmacological treatment,
+Phtime,current,optional,application time of pharmacological treatment,
+Sex,current,optional,Sex of the subject animal,
+Side,current,optional,Body side of the subject animal where measurements were made,
+CTV_ctv222,current,optional,value of ctv222 for this measurement,
+Odour,current,optional,Odorant (no comma) or list of odorants that are part of the stimulus (comma separated),
+StimONms,current,optional,"stimulus onset relative to start of recording in milliseconds, str or comma separated list",
+StimLen,current,optional,"stimulus lengths in milliseconds, str or comma separated list",
+Stimulus,current,optional,single name for stimulus applied (single odorant name or mix name),
+Measu,current,mandatory,"Number of the measurement (same as ""Measu"")",
+Animal,current,mandatory,name of the animal,
+Frame0,current,mandatory,"Value at frame 1 for this glomerulus (similarly Frame2, Frame3,….)",
+Frame1,current,mandatory,"Value at frame 2 for this glomerulus (similarly Frame2, Frame3,….)",
+NOdorNr,legacy,optional,integer code of odor,
+NRealTime,legacy,optional,time in minutes,
+NshiftX,legacy,optional,not used (was used in old IDL for shift correction),
+NshiftY,legacy,optional,not used (was used in old IDL for shift correction),
+NcontMeasu,legacy,optional,number of the control measurement. renamed as NControl,
+NodorN,legacy,optional,"no Measu for old setup, or slice for 3D measus new setup, 4th dimension in sig1 and sig1corr",
+Nstim2ON,legacy,optional,"stimulus onset of the second stimulus, frame (first frame WITH odor, frames are numbered 1, 2, ...)",
+Nstim2OFF,legacy,optional,"stimulus offset of the second stimulus, frame (last frame WITH odor, frames are numbered 1, 2, ...)",
+NAge,legacy,optional,exact age of the animal,replaced by TAge
+NAgeMax,legacy,optional,"maximum age of the animal, of exact age is not known. Replaced by TAge",
+TOdour,legacy,optional,"name of odor stimulus, e.g. 'HX1'. Replaced by TStimulus",
+Tcomment,legacy,optional,any comment. Replaced by TComment,
+Tanimal,legacy,optional,name of the animal. Replaced by TAnimal,
+T_dbb2,legacy,optional,raw data file name of the 380nm measurement for FURA.Replaced by TDBB2,
+CTV_ctv222,current,optional,value of ctv222 for this measurement,

view/flags_and_metadata_definitions/metadata_definition.csv

@@ -0,0 +1,43 @@
+LST Name,Type,Units,Settings Name,LSM metadata name,p1,Requirement for List load,Description,Default values
+Label,microscope,,label,"(“ScanInformation”, “Name)",ex_name,all,Measurement label as recoreded by Till Vision,not set yet
+Measu,microscope,,measu,,messungszahl,all,Measurement index as recoreded by Till Vision,0
+Analyze,analysis,,analyze,,analyze,all,"boolean flag, manually editable, that indicates whether or not the measurement is to be included in anaslysis",-1
+Odour,stimulus,,odorants,,odor,,Odorant (no comma) or list of odorants that are part of the stimulus (comma separated),not set yet
+Stimulus,stimulus,,stim,,stimulus,,single name for stimulus applied (single odorant name or mix name),not set yet
+OConc,stimulus,,conc,,odor_nr,,,not set yet
+DBB1,paths,,dbb,,dbb1,all,path of first raw recording file relative to the data folder,not set yet
+Lambda,microscope,nm,Lambda,"(“IlluminationChannels”, “Wavelength”)",lambda_nm,all,Monochromator wavelength,0
+StimON,stimulus,,stim_on,,,,onset in frames for first odorant (no comma) or multiple odorants (comma separated),-1
+StimONms,stimulus,,stim_on_ms,,,,onset in ms for first odorant (no comma) or multiple odorants (comma separated),-1
+StimOFF,stimulus,,stim_off,,,,offset in frames for first odorant (no comma) or multiple odorants (comma separated),-1
+StimLen,stimulus,,stim_len,,,,length in ms for first odorant (no comma) or multiple odorants (comma separated),0
+Stim2ON,stimulus,,stim2on,,,,onset in frames for second odorant (used only when timings of one odorant has been given above),0
+Stim2ONms,stimulus,,stim2on_ms,,,,onset in ms for second odorant (used only when timings of one odorant has been given above),-1
+Stim2OFF,stimulus,,stim2off,,,,offset in frames for second odorant (used only when timings of one odorant has been given above),0
+Stim2Len,stimulus,,stim2Len,,,,length in ms for second odorant (used only when timings of one odorant has been given above),0
+Cycle,recording,ms,GDMfreq,"(“TimeIntervall”,)",trial_ticks,all,Sampling Period of the Ca imaging video data for the measurement (1000/Sampling frequency),1E-06
+SampFreq,recording,Hz,SampFreq,,frequency,,Sampling frequency of measurement,1000000
+MTime,,,mtime,,m_time,,"time since first evidence that the animal is under the microscope, formatted as “HH:MM:SS.ssss”",“-00:00:00”
+Control,,,control,,kontrollmessung,,'Measu’ of the control measurement,0
+Pharma,treatment,,pharma,,treatment,,Pharmacological treatment applied,not set yet
+PhTime,treatment,,pharma_time,,pharma_time,,,0
+PhConc,treatment,,pharma_conc,,treat_conc,,Concentration of pharmacological treatment applied,0
+Comment,,,comment,,remark,,Arbitrary comment (most often contains information about the script used to generate measurement list),not set yet
+ShiftX,,,shiftx,,shiftX,,,0
+ShiftY,,,shifty,,shiftY,,,0
+StimISI,stimulus,,stim_isi,,stimulus_ISI,,,0
+setting,,,setting,,viewlabel,,,not set yet
+dbb2,paths,,dbb2,,dbb2,4,path of second raw recording file relative to the data folder,not set yet
+dbb3,paths,,dbb3,,dbb3,0,path of third raw recording file relative to the data folder,not set yet
+PxSzX,microscope,um,PxSzX,"(“VoxelSizeX”,)",pixelsizex,,Image resolution of Ca imaging video data along the X axis,0
+PxSzY,microscope,um,PxSzY,"(“VoxelSizeY”,)",pixelsizey,,Image resolution of Ca imaging video data along the Y axis,0
+PosZ,microscope,um,position_z,,posz,,depth of Z plane??,0
+Countl,,,countl,,,,,0
+slvFlip,,,slvflip,,,,,0
+Age,animal,days,age,,agetxt,,age of the subject animal in days(exact number or a range. e.g 2-4),-1
+Sex,animal,,sex,,sex,,Sex of the subject animal,not set yet
+Side,animal,,side,,bodyside,,Body side of the subject animal where measurements were made,not set yet
+UTC,recording,,UTC,"('Sample0time',)",utc,,Posix time stamp of measurement start,0
+FrameSizeX,microscope,pixels,FrameSizeX,,format_x,,Number of pixel in acquisition frame along X,0
+FrameSizeY,microscope,pixels,FrameSizeY,,format_y,,Number of pixel in acquisition frame along Y,0
+NumFrames,recording,,Nframes,,frames,,Number of frames measured,0

view/flags_and_metadata_definitions/setup_definitions.csv

@@ -0,0 +1,5 @@
+LE_loadExp,Description
+3,"Till Photonics, Single Wavelength"
+4,"Till Photonics, Dual Wavelength"
+20,"Zeis, single Wavelength"
+33,"VIEW-tif"

view/flags_and_metadata_definitions/view_flags_definition.csv

@@ -0,0 +1,243 @@
+Flag Name,Flag Subgroup,Flag Description,Selectable Options,Flag Value Type,Flag Default Value,Flag Checks,Error Message
+Data_Median_Filter,Filters,integer indicating the way in which the raw data should be filtered with a rolling window MEDIAN filter. Used in view/python_core/p1_class/filters.py,"0: no filter
+1: filter in space with size 3, no time filter
+2: filter in time with size 3, no space filter
+3: filter in space and time, using flag values (Data_Median_Filter_space)(Data_Median_Filter_time)",int,0,"{flag} in (0, 1, 2, 3)","Invalid value {flag} for {flag_name}, valid values are:\n0 (no median filtering)\n1 (median filtering only in space, with fixed window size)\n2 (median filtering only in time, with fixed window size)\n3 (median filtering in both space and time, with window sizes specified in the flags “Data_Median_Filter_space” and “Data_Median_Filter_time” respectively)"
+Data_Median_Filter_space,Filters,integer indicating width of the median filter used for filtering values over SPACE. Only used if Data_Median_Filter is set to 3.,,int,3,0 <= {flag} <= 3,"Invalid value {flag} for {flag_name}, valid values are 0, 1, 2 and 3"
+Data_Median_Filter_time,Filters,integer indicating width of the median filter used for filtering values over TIME. Only used if Data_Median_Filter is set to 3.,,int,3,0 <= {flag} <= 3,"Invalid value {flag} for {flag_name}, valid values are 0, 1, 2 and 3"
+Data_Mean_Filter,Filters,integer indicating whether the raw data should be filtered with a rolling window MEAN filter. Used in view/python_core/p1_class/filters.py,"0: no filter
+1: filter in space with size 3, no time filter
+2: filter in time with size 3, no space filter
+3: filter in space and time, using flag values (Data_Mean_Filter_space)(Data_Mean_Filter_time)",int,0,"{flag} in (0, 1, 2, 3)","Invalid value {flag} for {flag_name}, valid values are:\n0 (no median filtering)\n1 (median filtering only in space, with fixed window size)\n2 (median filtering only in time, with fixed window size)\n3 (median filtering in both space and time, with window sizes specified in the flags “Data_Median_Filter_space” and “Data_Median_Filter_time” respectively)"
+Data_Mean_Filter_space,Filters,integer indicating width of the mean filter used for filtering values over SPACE. Only used if Data_Mean_Filter is set to 3.,,int,3,{flag} >= 0,"Invalid value {flag} for {flag_name}, only non-negative integers are valid."
+Data_Mean_Filter_time,Filters,integer indicating width of the mean filter used for filtering values over TIME. Only used if Data_Mean_Filter is set to 3.,,int,3,{flag} >= 0,"Invalid value {flag} for {flag_name}, only non-negative integers are valid."
+LE_BleachCorrMethod,Filters,indicates the method for correcting bleaching artifacts.,"None: bleaching artifacts are not corrected
+log_uniform: fluorescence relative to background is assumed to follow the same exponential function for all pixels.
+log_pixelwise: fluorescence relative to background is assumed to follow different exponential functions for different pixels",str,None,,
+LE_BleachCutBorder,Filters,indicates the percentage of pixels to exclude along the border when not using AREA file,,float,20,,
+LE_BleachExcludeArea,Filters,"if True, function fitting for bleach correction will exclude pixels outside AREA mask, if an appropriate AREA file is found",,bool,True,,
+LE_BleachExcludeStimulus,Filters,"if True, function fitting for bleach correction will exclude stimulus frames. Stimulus frames are defined to begin at the end of background and they end based on LELog_ExcludeSeconds. Note that the end of the background depends on background flags",,bool,True,,
+LE_BleachStartFrame,Filters,"integer, for logarithmic bleach correction, all frames smaller are excluded in the fit function (frames are numbered 0, 1, 2...); used in CalcSigAll3000; default 2",,int,2,,
+LELog_ExcludeSeconds,Filters,"for logarithmic bleach correction, how many seconds after end of background to exclude. Note that background is defined by the flags in the subgroup BG-Background",,int,0,,
+LELog_InitialFactor,Filters,"for logarithmic bleach correction, all frames before stimulus onset are more important by this factor",,int,1,,
+LE_ScatteredLightFactor,Filters,strength of the scattered light correction to be applied. 0 turns off correction. 1 applied full strength correction,,float,0,0 <= {flag} <= 1,"Invalid value {flag} for {flag_name},valid values are in [0, 1]."
+LE_ScatteredLightRadius,Filters,radius of the scattered light correction to be applied in micrometers,,float,50,,
+LE_DefaultBackgroundRange,CalcSignals,specifies the default range of frame for background when stimulus information is not available or when LE_StimulusBasedBackground is set to False,,tuple,"(3,5)",,
+LE_PrestimEndBackground,CalcSignals,"integer, indicating the number of frames to exclude before stimulus onset when calculating background. Default: 2. background is calculated from LE_StartBackground to StimulusOn – LE_PrestimEndBackground - 1",,int,2,,
+LE_StartBackground,CalcSignals,"integer, indicating the number of frames to exclude from the beginning of a measurement when calculating background F for deltaF/F;set to -1 not to subtract background in data calculation, to frame for background start. Default: 4",,int,4,,
+LE_StimulusBasedBackground,CalcSignals,"bool, specifies whether stimulus info must be used to calculate background frames",,bool,True,,
+VIEW_CorrSignals,Xtra,set to access corrected buffer,,bool,0,,
+VIEW_InitCorr,Xtra,For corrected data (e.g. “subtract air”).,,int,0,,
+GDM_chunkPostStim,Output,Only used when the flag 'gdm_outputType' is 'chunks_only'. Indicates the number of seconds between stimulus offset and chunk end,,float,0,,
+GDM_chunkPreStim,Output,Only used when the flag 'gdm_outputType' is 'chunks_only'. Indicates the number of seconds between chunk start and stimulus onset,,float,0,,
+GDM_outputType,Output,kind of out to write into GDM file.,"
+full_traces: for each glomerulus/neural unit, add a row containing metadata + averaged trace
+chunks_only: for each glomerulus/neural unit and stimulus, add a row containing metadata + averaged trace",str,"full_traces",,
+GDM_withinArea,Output,whether to constrain and revise ROI masks to be within area when exporting GDM files,,bool,0,,
+LE_labelColumns,LoadData,"columns of the measurement list that will be concatenated to the internal label of measurements in VIEW.
+For possible column names, see the entries of the column 'LST Name' in the file `view/flags_and_metadata_definitions/metadata_definition.csv` of the repository",,tuple,"('Measu',)",,
+LE_loadExp,LoadData,integer indicating the setup used for acquiring data.,"0 : old setup
+1: Visicam
+2: confocal
+3: TILL photonics single wavelength
+4: TILL photonics dual wavelength (FURA)
+20: ZEISS multiphoton data
+33: Generic single wavelength TIF (TYX formatted data)
+34: Dual wavelength TIF in two separate files (TYX formatted data)
+35: Dual wavelength TIF in two separate files (TWYX formatted data; W=wavelength)
+665: generate test data(1)
+666: generate test data(2)
+667: generate test data(3)",int,3,,
+mv_bgColor,Movie,string indicating the color of border of the output movie. Valid values are those which can be interpreted as matplotlib colors,,str,k,,
+mv_bitrate,Movie,"string indicating the bitrate for exporting movies. E.g.: 100k is 100 kilobits/s, 1M is 1 megabits/s",,str,1024k,,
+mv_correctStimulusOnset,Movie,"time delay between stimulus time and imaging time. If more than 1000 interpreted as ms, else as frames",,int,0,,
+mv_cutborder,Movie,specifies the number of pixels to exclude on all sides of of the movie. Default value=0,,int,0,{flag} >= 0,"{flag_name} has to be >= 0, value specified = {flag}"
+mv_displayTime,Movie,"float controlling the font size of the time string in movie output. NOTE: Since all other fonts try to scale with the time string, this could be used to control the font size of all text in movie output.","0: no time string is drawn.
+1: time string is drawn, with an internally calculated appropriate font.
+(0, 1): try to make the font smaller than the internally calculated appropriate font. May not work if font size<8
+>1: try to make the font larger than internally calculated appropriate font.",float,0.8,{flag} >= 0,"{flag_name} has to be >=0, value specified={flag}"
+mv_exportFormat,Movie,specifies the format for saving movie,"single_tif: exports to single layer tif file
+stack_tif: exports to multi layer tif file
+libx264: exports to a compressed MP4 file with H264 encoding (very little loss of quality).
+ayuv: export to an uncompressed AVI file with AYUV encoding",str,libx264,,
+mv_fgColor,Movie,string indicating the color of annotations to be added onto the border of the output movie. Valid values are those which can be interpreted as matplotlib colors,,str,w,,
+mv_FirstFrame,Movie,"integer>=0 indicating a frame number, all frames before this frame will be excluded in movie output. First frame has number 0. Negative values will result in exclusion of no frames at the beginning.",,int,0,,
+mv_fontName,Movie,string indicating the name of the system font to use for annotating movies. E.g.: 'DejeVu Sans'. Use 'None' for default font,"OpenSans-Regular: a font included with VIEW
+PixelOperator8: a font included with VIEW",str,None,,
+mv_indiScale3factor,Movie,"float, Defines the central area to use when mv_individualScale=3. If it is set to 0.2, the central area is the part of the frame that remains after excluding 20% of frame width on left and right; and 20% of frame height on the top and bottom.",,float,0.2,0 <= {flag} <= 0.5,"{flag_name} has to be in [0, 0.5], value specified = {flag}"
+mv_individualScale,Movie,"Specifies the method for choosing the dynamic range within which data is restricted before creating movie output. In other words, specifies the data values that are to correspond to the bottom-most and top-most color of the colormap. Valid values are `x` and `yx` (0-6, 10-16, 20-26)","0 lower end is set to the value of the flag 'SO_MV_scalemin' and the upper end to the value of the flag 'SO_MV_scalemax'
+1: same as 0
+2: lower end is set to the minimum value of the entire movie, where 'minimum' is interpreted based on the the flag 'mv_percentileScale'. Similarly, the upper end is set to the maximum value of the entire movie.
+3: Similar to when 2, but the minimum and maximum values are calculated using a central region of the movie defined by the flag 'mv_indiScale3factor'
+4: lower end is set to the flag 'SO_MV_scalemin', upper end is set as when 2.
+5: lower end is set to the minimum of the region defined by the mask contained in the associated '.Area' or '.area.tif' file in the folder 'Coor'. Similarly the upper end is set to the maximum of the region.
+6: lower end is set to the value of the flag 'SO_MV_scalemin', the upper end is set as in when 5.
+11: Same as 1 above, but with the ranges minimum to 0 and 0 to maximum separately being mapped into lower and upper half range. Similarly 12, 13...
+21: Same as 1 above, but with center scaling at 0, adapt minimum or maximum accordingly. Similarly 22, 23..",int,0,,
+mv_LastFrame,Movie,integer>0 indicating a frame number. All frames after this frame will be excluded in movie output. First frame has number 0. Negative value or 0 will result in exclusion of no frames at the end.,,int,-1,,
+mv_lowerThreshPositiveResps,Movie,"string indicating a lower threshold, which is applied to the data indicated in 'mv_thresholdOn'. Pixels with values higher than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value higher than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 34.56%-100% of the data range will be colorized.
+Ignored if 'mv_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+mv_markStimulus,Movie,integer specifying the kind of stimulus marking to include.,"0: no marking
+1: for a square filled with foreground color (mv_fgColor)
+2: for a text containing odor and concentration information, as specified in measurement list
+3: for a red square
+21: for a text containing only odor information, as specified in measurement list",int,0,,
+mv_percentileScale,Movie,indicates if percentile values based on mv_percentileValue should be used instead on minimum and maximum while scaling data for movie export,,bool,False,,
+mv_percentileValue,Movie,"float, indicating a percentile value. Only used if mv_percentileScale is True. Data will be scaled to map the value at 'mv_percentileScale'th percentile to 0 and '100-mv_percentileScale'th percentile to 1",,float,0,0 <= {flag} <= 100,Invalid value {flag} for {flag_name}. Valid values are floats from b0 to 100
+mv_reverseIt,Movie,Specifies whether to flip vertically (i.e. left-right) each movie frame. Is identical to mv_rotateImage eq 7.,,bool,False,,
+mv_rotateImage,Movie,"Specifies the rotation to be applied to each movie frame. When set to 0, no rotation is applied. This flag emulates the IDL command 'rotate'.","0: no rotation
+1: 90cw
+2: 180cw
+3: 270cw
+4: flip and 90ccw
+5: flip and 180ccw
+6: flip and 270ccw
+7: flip with horizontal axis symmetry (i.e. up-down)",int,0,,
+mv_scaleLegendFactor,Movie,factor by which the upper and lower limits of the data are multiplied before being printed above and below the colorbar,,float,1,,
+mv_showROIs,Movie,Overlay other information to movie output. Valid values are 0 or integers greater than 10.,"0: do nothing
+10: puts unfilled squares/circles/polygons with the foreground color specified in the flag 'mv_fgColor'. ROI information sourced from a .COOR file.
+11: similar to 10, but ROI information is sourced from a .TIF file (IDL format). Similarly 12, 13.... See documentation of RM_ROITrace
+20: ROI information sourced as in 10, but the squares/circles/polygons have different colors and the overview frame underneath is replaced by all background.
+21: ROI information sourced as in 11, but ROIs drawn as in 20",int,0,,
+mv_SpeedFactor,Movie,"Specifies the factor by which the output movie is to be spedup. When set to 1, the frame rate of the output movie will be the same as during camera acquisition. When set to 2, twice as fast and when set to 0.5, half as fast.",,float,1,{flag} > 0,"{flag_name} has to be > 0, value specified = {flag}"
+mv_suppressMilliseconds,Movie,"if true, milliseconds will be excluded when printing time on movie frames",,bool,False,,
+mv_thresholdOn,Movie,string indicating the variable that is thresholded to decide the region of each frame that is colorized.,"none: all pixels will be colorized, subject to the flag 'mv_withinArea'. This is the default setting.
+raw1: raw data will be used
+sig1: signal data calculated from raw data will be used
+foto1: foto data calculated from raw data will be used",str,none,,
+mv_thresholdScale,Movie,string indicating how data outside the region of focus is to be scaled and mapped to gray values,"full: Black and white are respectively mapped onto the minimum and maximum of the movie data indicated by 'mv_thresholdShowImage'
+onlyShown: same as 'full', but min and max calculated restricted to the region outside the region of focus",str,full,,
+mv_thresholdShowImage,Movie,"string indicating the data to be shown outside the region of focus defined by 'mv_thresholdOn', 'mv_lowerThreshNegativeResps', 'mv_upperThreshNegativeResps' and 'mv_withinArea'.","foto1: foto data calculated from raw data will be used. This is the default setting. Since foto data is a frame, the region outside the region of focus will be the same for all frames
+raw1: raw data will be used.
+bgColor: all pixels outside the region of focus will be filled with the color indicated by the flag 'mv_bgColor'",str,foto1,,
+mv_upperThreshNegativeResps,Movie,"string indicating an upper threshold, which is applied to the data indicated in 'mv_thresholdOn'.
+Pixels with values lower than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value lower than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 0%-34.56% of the data range will be colorized.
+Ignored if 'mv_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+mv_withinArea,Movie,indicates whether SO_MV_colortable-based data colorization in movies is restricted to an area indicated by the `.area/.area.tif` file for this measurement/animal,,bool,0,,
+mv_xgap,Movie,"Specifies the width of the frame to be added for annnotations. If mv_xgap is 50, the frame will extend 50 pixels to the left of the data and 50 pixels to the right of the data.",,int,0,,
+mv_ygap,Movie,"Specifies the height of the frame to be added for annnotations. If mv_ygap is 50, the frame will extend 50 pixels above the data and 50 pixels below the data.",,int,0,,
+SO_bgColor,Output,string indicating the color of border of overviews and tapestries. Valid values are those which can be interpreted as matplotlib colors,,str,w,,
+SO_cutborder,Output,specifies the number of pixels to exclude on all sides of of the overview frame. Default value=0,,int,0,{flag} >= 0,"{flag_name} has to be >= 0, value specified = {flag}"
+SO_fgColor,Output,string indicating the color of annotations to be added onto to overviews and tapestries. Valid values are those which can be interpreted as matplotlib colors,,str,k,,
+SO_fontName,Output,string indicating the name of the system font to use for annotating overviews. E.g.: 'DejeVu Sans'. Use 'None' for default font,"OpenSans-Regular: a font included with VIEW
+PixelOperator8: a font included with VIEW",str,None,,
+SO_indiScale3factor,Output,"float, defines the central area to use when SO_individualScale=3. If it is set to 0.2, the central area is the part of the frame that remains after excluding 20% of frame width on left and right; and 20% of frame height on the top and bottom.",,float,0.2,0 <= {flag} <= 0.5,"{flag_name} has to be in [0, 0.5], value specified = {flag}"
+SO_individualScale,Output,"Specifies the method for choosing the dynamic range within which data is restricted before creating movie output. In other words, specifies the data values that are to correspond to the bottom-most and top-most color of the colormap. Valid values are `x` and `yx` (0-6, 10-16, 20-26)","0: lower end is set to the value of the flag 'SO_MV_scalemin' and the upper end to the value of the flag 'SO_MV_scalemax'
+1: same as 0
+2: lower end is set to the minimum value of the entire movie, where 'minimum' is interpreted based on the the flag 'mv_percentileScale'. Similarly, the upper end is set to the maximum value of the entire movie.
+3: Similar to when 2, but the minimum and maximum values are calculated using a central region of the movie defined by the flag 'mv_indiScale3factor'
+4: lower end is set to the flag 'SO_MV_scalemin', upper end is set as when 2.
+5: lower end is set to the minimum of the region defined by the mask contained in the associated '.Area' or '.area.tif' file in the folder 'Coor'. Similarly the upper end is set to the maximum of the region.
+6: lower end is set to the value of the flag 'SO_MV_scalemin', the upper end is set as in when 5.
+11: Same as 1 above, but with the ranges minimum to 0 and 0 to maximum separately being mapped into lower and upper half range. Similarly 12, 13...
+21: Same as 1 above, but with center scaling at 0, adapt minimum or maximum accordingly. Similarly 22, 23..",int,0,,
+SO_lowerThreshPositiveResps,Output,"string indicating a lower threshold, which is applied to the data indicated in 'SO_thresholdOn'. Pixels with values higher than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value higher than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 34.56%-100% of the data range will be colorized.
+Ignored if 'SO_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+SO_Method,Output,used in view/python_core/overviews/ctv_handlers.py. Indicates whether CTVs are to be calculated pixelwise or framewise.,"0: for calculations pixel by pixel (i.e. on the time-course in each pixel).
+10: for calculations frame by frame (much faster, but not all functions are possible).",int,0,,
+SO_percentileScale,Output,indicates if percentile values based on SO_percentileValue should be used instead on minimum and maximum while scaling data for generating overviews,,bool,False,,
+SO_percentileValue,Output,"float, indicating a percentile value. Only used if SO_percentileScale is True. Data will be scaled to map the value at 'SO_percentileScale'th percentile to the bottom-most color and '100-SO_percentileScale'th percentile to the top-most color.",,float,0,0 <= {flag} <= 100,Invalid value {flag} for {flag_name}. Valid values are floats from 0 to 100
+SO_reverseIt,Output,Specifies whether to flip vertically (i.e. left-right) the overview frame. Is identical to SO_rotateImage eq 7.,,bool,False,,
+SO_rotateImage,Output,"Specifies the rotation to be applied to each movie frame. When set to 0, no rotation is applied. This flag emulates the IDL command 'rotate'.","0: no rotation
+1: 90cw
+2: 180cw
+3: 270cw
+4: flip and 90ccw
+5: flip and 180ccw
+6: flip and 270ccw
+7: flip with horizontal axis symmetry (i.e. up-down)",int,0,,
+SO_scaleLegendFactor,Output,factor by which the upper and lower limits of the data are multiplied before being printed above and below the colorbar,,float,1,,
+SO_showROIs,Output,Overlay other information to overview output. Valid values are 0 or integers greater than 10.,"0: do nothing
+10: puts unfilled squares/circles/polygons with the foreground color specified in the flag 'mv_fgColor'. ROI information sourced from a .COOR file.
+11: similar to 10, but ROI information is sourced from a .TIF file (IDL format). Similarly 12, 13.... See documentation of RM_ROITrace
+20: ROI information sourced as in 10, but the squares/circles/polygons have different colors and the overview frame underneath is replaced by all background.
+21: ROI information sourced as in 11, but ROIs drawn as in 20",int,0,,
+SO_thresholdOn,Output,string indicating the variable that is thresholded to decide the region of each frame that is colorized.,"none: all pixels will be colorized, subject to the flag 'SO_withinArea'. This is the default setting.
+overview: overview image generated by applying CTV to sig1 is used
+foto1: foto data calculated from raw data will be used",str,none,,
+SO_thresholdScale,Output,string indicating how data outside the region of focus is to be scaled and mapped to gray values,"full: Black and white are respectively mapped onto the minimum and maximum of the movie data indicated by 'mv_thresholdShowImage'
+onlyShown: same as 'full', but min and max calculated restricted to the region outside the region of focus",str,full,,
+SO_thresholdShowImage,Output,"string indicating the data to be shown outside the region of focus defined by 'SO_thresholdOn', 'SO_lowerThreshNegativeResps', 'SO_upperThreshNegativeResps' and 'SO_withinArea'.","foto1: foto data calculated from raw data will be used. This is the default setting. Since foto data is a frame, the region outside the region of focus will be the same for all frames
+raw1: raw data will be used.
+bgColor: all pixels outside the region of focus will be filled with the color indicated by the flag 'mv_bgColor'",str,foto1,,
+SO_upperThreshNegativeResps,Output,"string indicating an upper threshold, which is applied to the data indicated in 'SO_thresholdOn'.
+Pixels with values lower than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value lower than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 0%-34.56% of the data range will be colorized.
+Ignored if 'mv_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+SO_withinArea,Output,indicates whether SO_MV_colortable-based data colorization in overviews is restricted to an area indicated by the `.area/.area.tif` file for this measurement/animal,,bool,0,,
+SO_xgap,Output,"integer, only used when 'CTV_scalebar' is True. Width of the padding added to the right and left f the overview frame is double this value",,int,30,,
+STG_MotherOfAllFolders,Paths,path on file system of the folder containing selected yml file,,str,not set yet,,
+STG_Datapath,Paths,"string pointing to a folder for raw data. Can be absolute or relative to STG_MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_Measu,Paths,measurement tag. Will be used to look up rows from LST files. Refers to the line with this value in column 'measu',,int,-1,,
+STG_OdorInfoPath,Paths,"string pointing to a folder for measurement list files. Can be absolute or relative to STG_MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_OdormaskPath,Paths,"string pointing to a folder for spatial footprint information of ROIs (primarily) and AREAs (as backup). Can be absolute or relative to STG_MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_OdorAreaPath,Paths,"string pointing to a folder for spatial footprint information of AREAs. Can be absolute or relative to STG_MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,Areas,,
+STG_OdorReportFile,Paths,string for free usage to define output file name,,str,empty,,
+STG_OdorReportPath,Paths,"string pointing to a folder for output (movies, overviews, tapestries, glodatamixes, pipeline reports). Can be absolute or relative to STG_MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_ReportTag,Paths,name of this animal e.g. in animal.lst,,str,not set yet,,
+STG_ProcessedDataPath,Paths,"string pointing to a folder for processed data (movement corrected movies, etc). Can be absolute or relative to STG_MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,Processed Data,,
+STG_TempArchivePath,Paths,string pointing to a folder which will be used to archive files generated for the comparison of pipelines. Must be absolute,,str,not set yet,,
+CTV_FeatureNumber,Output,"integer indicating which feature of the CTV to use, where CTV features are numbered 0, 1, 2, 3, ...",,int,0,{flag} >= 0,"Flag {flag_name} can only take integer values 0, 1, 2... Specified value: {flag}"
+CTV_firstframe,Output,"integer indicating a frame number, where frames are numbered 0, 1, 2...",,int,22,,
+CTV_lastframe,Output,"integer indicating a frame number, where frames are numbered 0, 1, 2...",,int,36,,
+CTV_Method,Output,"specifies curve-to-value function for pixelwise application to movies to generate overviews. To specify a custom-written ctv method, specify function name here and the file containing it in 'CTV_MethodFile'. Or use an internally implemented CTV","22: is the difference between two fixed points
+0: one feature, (definition taken forward from VIEW-IDL) mean of all frames, useful for simulated photographs
+22: one feature, (mean of 3 frames around <last_frame>) – (mean of 3 frames around <first_frame>)
+222: one feature, (mean of 4 frames starting at <last_frame>) – (mean of 4 frames starting at <first_frame>)
+35: one feature, relates to the maximum within 3 secs after stimulus onset
+22and35: two features, of 22 and 35 above
+300: one feature, same as 0
+301: one feature, (definition taken forward from VIEW-IDL) mean of frames 5 to 10, which is generally before stimulus onset. Useful for morphological views.
+302: one feature, mean of frames from <first_frame> to <last_frame> (both inclusive).
+303: one feature, Average of background frames, calculated using first stimulus onset and the flags LE_StartBackground and LE_PrestimEndBackground. Can be useful to visualize and compare baseline values of signals.
+330: one feature, median of all frames
+331: one feature, median of frames 5 to 10, which is generally before stimulus onset. Useful for morphological views.
+332: one feature, median of frames from <first_frame> to <last_frame> (both inclusive).
+333: one feature, Median of background frames, calculated using first stimulus onset and the flags LE_StartBackground and LE_PrestimEndBackground. Can be useful to visualize and compare baseline values of signals.","int, str",22,,
+CTV_MethodFile,Output,only used when 'CTV_Method',,str,not set yet,,
+CTV_scalebar,Output,"when True, colorbar is added to the right of overviews and movies",,bool,False,,
+CTV_StimulusNumber,Output,"integer indicating which stimulus to use when calculating CTV, where stimuli are numbered 0, 1, 2, 3, ...",,int,0,{flag} >= 0,"Flag {flag_name} can only take integer values 0, 1, 2... Specified value: {flag}"
+CTVM_Method,Output,(not yet implemented in pyVIEW)for multiple CTV values at once,,int,0,,
+Signal_FilterSpaceFlag,Output,Toggles on/off the filter applied to sig1 before generating overview frames or movies,,bool,0,,
+Signal_FilterSpaceSize,Output,standard deviation of the spatial gaussian filter applied to sig1 before creating overviews/movies. Only used if 'Signal_FilterSpaceFlag' is True,,float,3,,
+Signal_FilterTimeFlag,Output,Toggles on/off the filter applied to sig1 before generating movies,,bool,0,,
+Signal_FilterTimeSize,Output,standard deviation of the temporal gaussian filter applied to sig1 before creating movies. Only used if 'Signal_FilterTimeFlag' is True,,float,3,,
+RM_differentViews,Output,"Change view, e.g. mirror flip right ALs   (SingleOverviews.pro)",,bool,0,,
+RM_Radius,Output,"integer representing number of pixels, interpreted as the size of glomeruli read from .coor file, used when marking ROIs on overviews and exporting glodatamixes",,int,5,,
+RM_ROIThreshold,Output,"When converting spatial footprints of ROIs into binary masks, all pixel above this percentile will be considered to be inside the binary mask",,float,75,,
+RM_ROITrace,Output,Indicates the file from which component masks are sourced,"0 : read the .coor file
+1 : (not tested) read the .tif file (8-bit, value indicates the name of the glomerulus)
+2 : read the .Area file (IDL format).
+3 : read the .roi file created in ILTIS
+4 : read the .roi.tif file created by a processing pipeline
+5 : read the .area.tif file created in ILTIS
+6 : use a fictive ROI with uniform weight over the entire frame",int,3,,
+SO_MV_scalemax,Output,value to scale maximum to with SO_indivdiualScale or mv_individualScale equals 0,,float,1,,
+SO_MV_scalemin,Output,corresponding value for minimum,,float,0,,
+LE_CalcMethod,CalcSignals,integer indicating the method used for calculating a biologically relevant signal from raw movie data. Used in view/python_core/calc_methods.py,"
+0: sig1 = raw1 / 1000
+1: sig1 = raw2 / 1000
+3: sig1 = deltaF/F
+4: sig1 = raw1/raw2 - (raw1/raw2 averaged over the background)",int,3,,
+VIEW_batchmode,Xtra,"choice between interactive mode and VIEW_batchmode. Valid values are: True, False ",,bool,True,,
+SO_MV_colortable,Xtra,"if integer indicates an IDL_style color table to use. Valid values are: 11-14. Values are 12-14 self-programmed rainbows.
+if string, must be the name of a matplotlib colormap.
+See https://matplotlib.org/tutorials/colors/colormaps.html.
+NOTE: The inverse of every matplotlib colormap is also available, just append '_r' to colormap name. E.g.: Inverse of the colormap 'jet' would be 'jet_r'","
+11: a self programmed SO_MV_colortable
+12: a self-programmed rainbow SO_MV_colortable
+13: a self-programmed rainbow SO_MV_colortable
+14: a self-programmed rainbow SO_MV_colortable
+jet: a matplotlib colormap
+viridis: a matplotlib colormap, etc","int, str",13,,

view/flags_and_metadata_definitions/view_flags_renaming_2021.csv

@@ -0,0 +1,236 @@
+Flag Name,Flag Subgroup,Old Flag Name,Old Flag Subgroup,Flag Description,Selectable Options,Flag Value Type,Flag Default Value,Flag Checks,Error Message
+Data_Median_Filter,Filters,CSM_Median,ARCO-artifacts,integer indicating the way in which the raw data should be filtered with a rolling window MEDIAN filter. Used in view/python_core/p1_class/filters.py,"0: no filter
+1: filter in space with size 3, no time filter
+2: filter in time with size 3, no space filter
+3: filter in space and time, using flag values (CSM_Median_space)(CSM_Median_time)",int,0,"{flag} in (0, 1, 2, 3)","Invalid value {flag} for {flag_name}, valid values are:\n0 (no median filtering)\n1 (median filtering only in space, with fixed window size)\n2 (median filtering only in time, with fixed window size)\n3 (median filtering in both space and time, with window sizes specified in the flags “CSM_Median_space” and “CSM_Median_time” respectively)"
+Data_Median_Space,Filters,CSM_Median_space,ARCO-artifacts,integer indicating width of the median filter used for filtering values over SPACE. Only used if CSM_Median is set to 3.,,int,3,0 <= {flag} <= 3,"Invalid value {flag} for {flag_name}, valid values are 0, 1, 2 and 3"
+Data_Median_Time,Filters,CSM_Median_time,ARCO-artifacts,integer indicating width of the median filter used for filtering values over TIME. Only used if CSM_Median is set to 3.,,int,3,0 <= {flag} <= 3,"Invalid value {flag} for {flag_name}, valid values are 0, 1, 2 and 3"
+Data_Mean_Filter,Filters,CSM_Mean,ARCO-artifacts,integer indicating whether the raw data should be filtered with a rolling window MEAN filter. Used in view/python_core/p1_class/filters.py,"0: no filter
+1: filter in space with size 3, no time filter
+2: filter in time with size 3, no space filter
+3: filter in space and time, using flag values (CSM_Mean_space)(CSM_Mean_time)",int,0,"{flag} in (0, 1, 2, 3)","Invalid value {flag} for {flag_name}, valid values are:\n0 (no median filtering)\n1 (median filtering only in space, with fixed window size)\n2 (median filtering only in time, with fixed window size)\n3 (median filtering in both space and time, with window sizes specified in the flags “CSM_Median_space” and “CSM_Median_time” respectively)"
+Data_Mean_Space,Filters,CSM_Mean_space,ARCO-artifacts,integer indicating width of the mean filter used for filtering values over SPACE. Only used if CSM_Mean is set to 3.,,int,3,{flag} >= 0,"Invalid value {flag} for {flag_name}, only non-negative integers are valid."
+Data_Mean_Time,Filters,CSM_Mean_time,ARCO-artifacts,integer indicating width of the mean filter used for filtering values over TIME. Only used if CSM_Mean is set to 3.,,int,3,{flag} >= 0,"Invalid value {flag} for {flag_name}, only non-negative integers are valid."
+LE_BleachCorrMethod,LoadData,LE_BleachCorrMethod,ARCO-artifacts,indicates the method for correcting bleaching artifacts.,"None: bleaching artifacts are not corrected
+log_uniform: fluorescence relative to background is assumed to follow the same exponential function for all pixels.
+log_pixelwise: fluorescence relative to background is assumed to follow different exponential functions for different pixels",str,None,,
+LE_BleachCutBorder,LoadData,LE_BleachCutBorder,ARCO-artifacts,indicates the percentage of pixels to exclude along the border when not using AREA file,,float,20,,
+LE_BleachExcludeArea,LoadData,LE_BleachExcludeArea,ARCO-artifacts,"if True, function fitting for bleach correction will exclude pixels outside AREA mask, if an appropriate AREA file is found",,bool,True,,
+LE_BleachExcludeStimulus,LoadData,LE_BleachExcludeStimulus,ARCO-artifacts,"if True, function fitting for bleach correction will exclude stimulus frames. Stimulus frames are defined to begin at the end of background and they end based on LE_LogExcludeSeconds. Note that the end of the background depends on background flags",,bool,True,,
+LE_BleachStartFrame,LoadData,LE_BleachStartFrame,ARCO-artifacts,"integer, for logarithmic bleach correction, all frames smaller are excluded in the fit function (frames are numbered 0, 1, 2...); used in CalcSigAll3000; default 2",,int,2,,
+LELog_ExcludeSeconds,LoadData,LE_LogExcludeSeconds,ARCO-artifacts,"for logarithmic bleach correction, how many seconds after end of background to exclude",,int,0,,
+LELog_InitialFactor,LoadData,LE_LogInitialFactor,ARCO-artifacts,"for logarithmic bleach correction, all frames before stimulus onset are more important by this factor",,int,1,,
+LE_ScatteredLightFactor,LoadData,LE_ScatteredLightFactor,ARCO-artifacts,strength of the scattered light correction to be applied. 0 turns off correction. 1 applied full strength correction,,float,0,0 <= {flag} <= 1,"Invalid value {flag} for {flag_name},valid values are in [0, 1]."
+LE_ScatteredLightRadius,LoadData,LE_ScatteredLightRadius,ARCO-artifacts,radius of the scattered light correction to be applied in micrometers,,float,50,,
+LE_DefaultBackgroundRange,CalcSignals,LE_DefaultBackgroundRange,BG-background,specifies the default range of frame for background when stimulus information is not available or when LE_StimulusBasedBackground is set to False,,tuple,"(3,5)",,
+LE_PrestimEndBackground,CalcSignals,LE_PrestimEndBackground,BG-background,"integer, indicating the number of frames to exclude before stimulus onset when calculating background. Default: 2. background is calculated from LE_StartBackground to StimulusOn – LE_PrestimEndBackground - 1",,int,2,,
+LE_StartBackground,CalcSignals,LE_StartBackground,BG-background,"integer, indicating the number of frames to exclude from the beginning of a measurement when calculating background F for deltaF/F;set to -1 not to subtract background in data calculation, to frame for background start. Default: 4",,int,4,,
+LE_StimulusBasedBackground,CalcSignals,LE_StimulusBasedBackground,BG-background,"bool, specifies whether stimulus info must be used to calculate background frames",,bool,True,,
+VIEW_CorrSignals,Xtra,CorrectFlag,CORR-correction,set to access corrected buffer,,bool,0,,
+VIEW_InitCorr,Xtra,VIEW_InitCorr,CORR-correction,For corrected data (e.g. “subtract air”).,,int,0,,
+GDM_withinArea,Output,gdm_withinArea,GDM-gdm export,whether to constrain and revise ROI masks to be within area when exporting GDM files,,bool,0,,
+LE_labelColumns,LoadData,view_labelColumns,LE-loading data,"columns of the measurement list that will be concatenated to the internal label of measurements in VIEW.
+For possible column names, see the entries of the column 'LST Name' in the file `view/flags_and_metadata_definitions/metadata_definition.csv` of the repository",,tuple,"('Measu',)",,
+LE_loadExp,LoadData,view_loadExp,LE-loading data,integer indicating the setup used for acquiring data.,"0 : old setup
+1: Visicam
+2: confocal
+3: TILL photonics single wavelength
+4: TILL photonics dual wavelength (FURA)
+20: ZEISS multiphoton data
+33: Generic single wavelength TIF (TYX formatted data)
+34: Dual wavelength TIF in two separate files (TYX formatted data)
+35: Dual wavelength TIF in two separate files (TWYX formatted data; W=wavelength)
+665: generate test data(1)
+666: generate test data(2)
+667: generate test data(3)",int,3,,
+mv_bgColor,Movie,mv_bgColor,MV-movie,string indicating the color of border of the output movie. Valid values are those which can be interpreted as matplotlib colors,,str,k,,
+mv_bitrate,Movie,mv_bitrate,MV-movie,"string indicating the bitrate for exporting movies. E.g.: 100k is 100 kilobits/s, 1M is 1 megabits/s",,str,1024k,,
+mv_correctStimulusOnset,Movie,mv_correctStimulusOnset,MV-movie,"time delay between stimulus time and imaging time. If more than 1000 interpreted as ms, else as frames",,int,0,,
+mv_cutborder,Movie,mv_cutborder,MV-movie,specifies the number of pixels to exclude on all sides of of the movie. Default value=0,,int,0,{flag} >= 0,"{flag_name} has to be >= 0, value specified = {flag}"
+mv_displayTime,Movie,mv_displayTime,MV-movie,"float controlling the font size of the time string in movie output. NOTE: Since all other fonts try to scale with the time string, this could be used to control the font size of all text in movie output.","0: no time string is drawn.
+1: time string is drawn, with an internally calculated appropriate font.
+(0, 1): try to make the font smaller than the internally calculated appropriate font. May not work if font size<8
+>1: try to make the font larger than internally calculated appropriate font.",float,0.8,{flag} >= 0,"{flag_name} has to be >=0, value specified={flag}"
+mv_exportFormat,Movie,mv_exportFormat,MV-movie,specifies the format for saving movie,"single_tif: exports to single layer tif file
+stack_tif: exports to multi layer tif file
+libx264: exports to a compressed MP4 file with H264 encoding (very little loss of quality).
+ayuv: export to an uncompressed AVI file with AYUV encoding",str,libx264,,
+mv_fgColor,Movie,mv_fgColor,MV-movie,string indicating the color of annotations to be added onto the border of the output movie. Valid values are those which can be interpreted as matplotlib colors,,str,w,,
+mv_FirstFrame,Movie,mv_FirstFrame,MV-movie,"integer>=0 indicating a frame number, all frames before this frame will be excluded in movie output. First frame has number 0. Negative values will result in exclusion of no frames at the beginning.",,int,0,,
+mv_fontName,Movie,mv_fontName,MV-movie,string indicating the name of the system font to use for annotating movies. E.g.: 'DejeVu Sans'. Use 'None' for default font,"OpenSans-Regular: a font included with VIEW
+PixelOperator8: a font included with VIEW",str,None,,
+mv_indiScale3factor,Movie,mv_indiScale3factor,MV-movie,"float, Defines the central area to use when mv_individualScale=3. If it is set to 0.2, the central area is the part of the frame that remains after excluding 20% of frame width on left and right; and 20% of frame height on the top and bottom.",,float,0.2,0 <= {flag} <= 0.5,"{flag_name} has to be in [0, 0.5], value specified = {flag}"
+mv_individualScale,Movie,mv_individualScale,MV-movie,"Specifies the method for choosing the dynamic range within which data is restricted before creating movie output. In other words, specifies the data values that are to correspond to the bottom-most and top-most color of the colormap. Valid values are `x` and `yx` (0-6, 10-16, 20-26)","0 lower end is set to the value of the flag 'scalemin' and the upper end to the value of the flag 'scalemax'
+1: same as 0
+2: lower end is set to the minimum value of the entire movie, where 'minimum' is interpreted based on the the flag 'mv_percentileScale'. Similarly, the upper end is set to the maximum value of the entire movie.
+3: Similar to when 2, but the minimum and maximum values are calculated using a central region of the movie defined by the flag 'mv_indiScale3factor'
+4: lower end is set to the flag 'scalemin', upper end is set as when 2.
+5: lower end is set to the minimum of the region defined by the mask contained in the associated '.Area' or '.area.tif' file in the folder 'Coor'. Similarly the upper end is set to the maximum of the region.
+6: lower end is set to the value of the flag 'scalemin', the upper end is set as in when 5.
+11: Same as 1 above, but with the ranges minimum to 0 and 0 to maximum separately being mapped into lower and upper half range. Similarly 12, 13...
+21: Same as 1 above, but with center scaling at 0, adapt minimum or maximum accordingly. Similarly 22, 23..",int,0,,
+mv_LastFrame,Movie,mv_LastFrame,MV-movie,integer>0 indicating a frame number. All frames after this frame will be excluded in movie output. First frame has number 0. Negative value or 0 will result in exclusion of no frames at the end.,,int,-1,,
+mv_lowerThreshPositiveResps,Movie,mv_lowerThreshPositiveResps,MV-movie,"string indicating a lower threshold, which is applied to the data indicated in 'mv_thresholdOn'. Pixels with values higher than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value higher than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 34.56%-100% of the data range will be colorized.
+Ignored if 'mv_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+mv_markStimulus,Movie,mv_markStimulus,MV-movie,integer specifying the kind of stimulus marking to include.,"0: no marking
+1: for a square filled with foreground color (mv_fgColor)
+2: for a text containing odor and concentration information, as specified in measurement list
+3: for a red square
+21: for a text containing only odor information, as specified in measurement list",int,0,,
+mv_percentileScale,Movie,mv_percentileScale,MV-movie,indicates if percentile values based on mv_percentileValue should be used instead on minimum and maximum while scaling data for movie export,,bool,False,,
+mv_percentileValue,Movie,mv_percentileValue,MV-movie,"float, indicating a percentile value. Only used if mv_percentileScale is True. Data will be scaled to map the value at 'mv_percentileScale'th percentile to 0 and '100-mv_percentileScale'th percentile to 1",,float,0,0 <= {flag} <= 100,Invalid value {flag} for {flag_name}. Valid values are floats from b0 to 100
+mv_reverseIt,Movie,mv_reverseIt,MV-movie,Specifies whether to flip vertically (i.e. left-right) each movie frame. Is identical to mv_rotateImage eq 7.,,bool,False,,
+mv_rotateImage,Movie,mv_rotateImage,MV-movie,"Specifies the rotation to be applied to each movie frame. When set to 0, no rotation is applied. This flag emulates the IDL command 'rotate'.","0: no rotation
+1: 90cw
+2: 180cw
+3: 270cw
+4: flip and 90ccw
+5: flip and 180ccw
+6: flip and 270ccw
+7: flip with horizontal axis symmetry (i.e. up-down)",int,0,,
+mv_scaleLegendFactor,Movie,mv_scaleLegendFactor,MV-movie,factor by which the upper and lower limits of the data are multiplied before being printed above and below the colorbar,,float,1,,
+mv_showROIs,Movie,mv_showROIs,MV-movie,Overlay other information to movie output. Valid values are 0 or integers greater than 10.,"0: do nothing
+10: puts unfilled squares/circles/polygons with the foreground color specified in the flag 'mv_fgColor'. ROI information sourced from a .COOR file.
+11: similar to 10, but ROI information is sourced from a .TIF file (IDL format). Similarly 12, 13.... See documentation of RM_ROITrace
+20: ROI information sourced as in 10, but the squares/circles/polygons have different colors and the overview frame underneath is replaced by all background.
+21: ROI information sourced as in 11, but ROIs drawn as in 20",int,0,,
+mv_SpeedFactor,Movie,mv_SpeedFactor,MV-movie,"Specifies the factor by which the output movie is to be spedup. When set to 1, the frame rate of the output movie will be the same as during camera acquisition. When set to 2, twice as fast and when set to 0.5, half as fast.",,float,1,{flag} > 0,"{flag_name} has to be > 0, value specified = {flag}"
+mv_suppressMilliseconds,Movie,mv_suppressMilliseconds,MV-movie,"if true, milliseconds will be excluded when printing time on movie frames",,bool,False,,
+mv_thresholdOn,Movie,mv_thresholdOn,MV-movie,string indicating the variable that is thresholded to decide the region of each frame that is colorized.,"none: all pixels will be colorized, subject to the flag 'mv_withinArea'. This is the default setting.
+raw1: raw data will be used
+sig1: signal data calculated from raw data will be used
+foto1: foto data calculated from raw data will be used",str,none,,
+mv_thresholdScale,Movie,mv_thresholdScale,MV-movie,string indicating how data outside the region of focus is to be scaled and mapped to gray values,"full: Black and white are respectively mapped onto the minimum and maximum of the movie data indicated by 'mv_thresholdShowImage'
+onlyShown: same as 'full', but min and max calculated restricted to the region outside the region of focus",str,full,,
+mv_thresholdShowImage,Movie,mv_thresholdShowImage,MV-movie,"string indicating the data to be shown outside the region of focus defined by 'mv_thresholdOn', 'mv_lowerThreshNegativeResps', 'mv_upperThreshNegativeResps' and 'mv_withinArea'.","foto1: foto data calculated from raw data will be used. This is the default setting. Since foto data is a frame, the region outside the region of focus will be the same for all frames
+raw1: raw data will be used.
+bgColor: all pixels outside the region of focus will be filled with the color indicated by the flag 'mv_bgColor'",str,foto1,,
+mv_upperThreshNegativeResps,Movie,mv_upperThreshNegativeResps,MV-movie,"string indicating an upper threshold, which is applied to the data indicated in 'mv_thresholdOn'.
+Pixels with values lower than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value lower than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 0%-34.56% of the data range will be colorized.
+Ignored if 'mv_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+mv_withinArea,Movie,mv_withinArea,MV-movie,indicates whether colortable-based data colorization in movies is restricted to an area indicated by the `.area/.area.tif` file for this measurement/animal,,bool,0,,
+mv_xgap,Movie,mv_xgap,MV-movie,"Specifies the width of the frame to be added for annnotations. If mv_xgap is 50, the frame will extend 50 pixels to the left of the data and 50 pixels to the right of the data.",,int,0,,
+mv_ygap,Movie,mv_ygap,MV-movie,"Specifies the height of the frame to be added for annnotations. If mv_ygap is 50, the frame will extend 50 pixels above the data and 50 pixels below the data.",,int,0,,
+SO_bgColor,Output,SO_bgColor,SO-overviews,string indicating the color of border of overviews and tapestries. Valid values are those which can be interpreted as matplotlib colors,,str,w,,
+SO_cutborder,Output,SO_cutborder,SO-overviews,specifies the number of pixels to exclude on all sides of of the overview frame. Default value=0,,int,0,{flag} >= 0,"{flag_name} has to be >= 0, value specified = {flag}"
+SO_fgColor,Output,SO_fgColor,SO-overviews,string indicating the color of annotations to be added onto to overviews and tapestries. Valid values are those which can be interpreted as matplotlib colors,,str,k,,
+SO_fontName,Output,SO_fontName,SO-overviews,string indicating the name of the system font to use for annotating overviews. E.g.: 'DejeVu Sans'. Use 'None' for default font,"OpenSans-Regular: a font included with VIEW
+PixelOperator8: a font included with VIEW",str,None,,
+SO_indiScale3factor,Output,SO_indiScale3factor,SO-overviews,"float, defines the central area to use when SO_individualScale=3. If it is set to 0.2, the central area is the part of the frame that remains after excluding 20% of frame width on left and right; and 20% of frame height on the top and bottom.",,float,0.2,0 <= {flag} <= 0.5,"{flag_name} has to be in [0, 0.5], value specified = {flag}"
+SO_individualScale,Output,SO_individualScale,SO-overviews,"Specifies the method for choosing the dynamic range within which data is restricted before creating movie output. In other words, specifies the data values that are to correspond to the bottom-most and top-most color of the colormap. Valid values are `x` and `yx` (0-6, 10-16, 20-26)","0: lower end is set to the value of the flag 'scalemin' and the upper end to the value of the flag 'scalemax'
+1: same as 0
+2: lower end is set to the minimum value of the entire movie, where 'minimum' is interpreted based on the the flag 'mv_percentileScale'. Similarly, the upper end is set to the maximum value of the entire movie.
+3: Similar to when 2, but the minimum and maximum values are calculated using a central region of the movie defined by the flag 'mv_indiScale3factor'
+4: lower end is set to the flag 'scalemin', upper end is set as when 2.
+5: lower end is set to the minimum of the region defined by the mask contained in the associated '.Area' or '.area.tif' file in the folder 'Coor'. Similarly the upper end is set to the maximum of the region.
+6: lower end is set to the value of the flag 'scalemin', the upper end is set as in when 5.
+11: Same as 1 above, but with the ranges minimum to 0 and 0 to maximum separately being mapped into lower and upper half range. Similarly 12, 13...
+21: Same as 1 above, but with center scaling at 0, adapt minimum or maximum accordingly. Similarly 22, 23..",int,0,,
+SO_lowerThreshPositiveResps,Output,SO_lowerThreshPositiveResps,SO-overviews,"string indicating a lower threshold, which is applied to the data indicated in 'SO_thresholdOn'. Pixels with values higher than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value higher than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 34.56%-100% of the data range will be colorized.
+Ignored if 'SO_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+SO_Method,Output,SO_Method,SO-overviews,used in view/python_core/overviews/ctv_handlers.py. Indicates whether CTVs are to be calculated pixelwise or framewise.,"0: for calculations pixel by pixel (i.e. on the time-course in each pixel).
+10: for calculations frame by frame (much faster, but not all functions are possible).",int,0,,
+SO_percentileScale,Output,SO_percentileScale,SO-overviews,indicates if percentile values based on SO_percentileValue should be used instead on minimum and maximum while scaling data for generating overviews,,bool,False,,
+SO_percentileValue,Output,SO_percentileValue,SO-overviews,"float, indicating a percentile value. Only used if SO_percentileScale is True. Data will be scaled to map the value at 'SO_percentileScale'th percentile to the bottom-most color and '100-SO_percentileScale'th percentile to the top-most color.",,float,0,0 <= {flag} <= 100,Invalid value {flag} for {flag_name}. Valid values are floats from 0 to 100
+SO_reverseIt,Output,SO_reverseIt,SO-overviews,Specifies whether to flip vertically (i.e. left-right) the overview frame. Is identical to SO_rotateImage eq 7.,,bool,False,,
+SO_rotateImage,Output,SO_rotateImage,SO-overviews,"Specifies the rotation to be applied to each movie frame. When set to 0, no rotation is applied. This flag emulates the IDL command 'rotate'.","0: no rotation
+1: 90cw
+2: 180cw
+3: 270cw
+4: flip and 90ccw
+5: flip and 180ccw
+6: flip and 270ccw
+7: flip with horizontal axis symmetry (i.e. up-down)",int,0,,
+SO_scaleLegendFactor,Output,SO_scaleLegendFactor,SO-overviews,factor by which the upper and lower limits of the data are multiplied before being printed above and below the colorbar,,float,1,,
+SO_showROIs,Output,SO_showROIs,SO-overviews,Overlay other information to overview output. Valid values are 0 or integers greater than 10.,"0: do nothing
+10: puts unfilled squares/circles/polygons with the foreground color specified in the flag 'mv_fgColor'. ROI information sourced from a .COOR file.
+11: similar to 10, but ROI information is sourced from a .TIF file (IDL format). Similarly 12, 13.... See documentation of RM_ROITrace
+20: ROI information sourced as in 10, but the squares/circles/polygons have different colors and the overview frame underneath is replaced by all background.
+21: ROI information sourced as in 11, but ROIs drawn as in 20",int,0,,
+SO_thresholdOn,Output,SO_thresholdOn,SO-overviews,string indicating the variable that is thresholded to decide the region of each frame that is colorized.,"none: all pixels will be colorized, subject to the flag 'SO_withinArea'. This is the default setting.
+overview: overview image generated by applying CTV to sig1 is used
+foto1: foto data calculated from raw data will be used",str,none,,
+SO_thresholdScale,Output,SO_thresholdScale,SO-overviews,string indicating how data outside the region of focus is to be scaled and mapped to gray values,"full: Black and white are respectively mapped onto the minimum and maximum of the movie data indicated by 'mv_thresholdShowImage'
+onlyShown: same as 'full', but min and max calculated restricted to the region outside the region of focus",str,full,,
+SO_thresholdShowImage,Output,SO_thresholdShowImage,SO-overviews,"string indicating the data to be shown outside the region of focus defined by 'SO_thresholdOn', 'SO_lowerThreshNegativeResps', 'SO_upperThreshNegativeResps' and 'SO_withinArea'.","foto1: foto data calculated from raw data will be used. This is the default setting. Since foto data is a frame, the region outside the region of focus will be the same for all frames
+raw1: raw data will be used.
+bgColor: all pixels outside the region of focus will be filled with the color indicated by the flag 'mv_bgColor'",str,foto1,,
+SO_upperThreshNegativeResps,Output,SO_upperThreshNegativeResps,SO-overviews,"string indicating an upper threshold, which is applied to the data indicated in 'SO_thresholdOn'.
+Pixels with values lower than this threshold will be colorized.
+Absolute and relative thresholds can be specified for this flag.
+For absolute thresholds use the format 'axxxx'. E.g.: 'a956.56', in which case all pixels with value lower than 956.56 will be colorized.
+For relative threholds use the format 'ryyyy'. E.g.: 'r34.56', in which case all pixels with values in the range 0%-34.56% of the data range will be colorized.
+Ignored if 'mv_thresholdOn' is 'none'",,str,a0,{flag}.startswith('a') or {flag}.startswith('r'),Invalid value {flag} specified for {flag_name}! Use the format 'rxxxx' for relative values and 'ayyyy' for absolute values
+SO_withinArea,Output,SO_withinArea,SO-overviews,indicates whether colortable-based data colorization in overviews is restricted to an area indicated by the `.area/.area.tif` file for this measurement/animal,,bool,0,,
+SO_xgap,Output,SO_xgap,SO-overviews,"integer, only used when 'CTV_scalebar' is True. Width of the padding added to the right and left f the overview frame is double this value",,int,30,,
+STG_MotherOfAllFolders,Paths,MotherOfAllFolders,PTH-paths,path on file system of the folder containing selected yml file,,str,not set yet,,
+STG_Datapath,Paths,STG_Datapath,PTH-paths,"string pointing to a folder for raw data. Can be absolute or relative to MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_Measu,Paths,STG_Measu,PTH-paths,measurement tag. Will be used to look up rows from LST files. Refers to the line with this value in column 'measu',,int,-1,,
+STG_OdorInfoPath,Paths,STG_OdorInfoPath,PTH-paths,"string pointing to a folder for measurement list files. Can be absolute or relative to MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_OdormaskPath,Paths,STG_OdormaskPath,PTH-paths,"string pointing to a folder for spatial footprint information of ROIs (primarily) and AREAs (as backup). Can be absolute or relative to MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_OdorAreaPath,Paths,STG_OdorAreaPath,PTH-paths,"string pointing to a folder for spatial footprint information of AREAs. Can be absolute or relative to MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,Areas,,
+STG_OdorReportFile,Paths,STG_OdorReportFile,PTH-paths,string for free usage to define output file name,,str,empty,,
+STG_OdorReportPath,Paths,STG_OdorReportPath,PTH-paths,"string pointing to a folder for output (movies, overviews, tapestries, glodatamixes, pipeline reports). Can be absolute or relative to MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,not set yet,,
+STG_ReportTag,Paths,STG_ReportTag,PTH-paths,name of this animal e.g. in animal.lst,,str,not set yet,,
+STG_ProcessedDataPath,Paths,STG_ProcessedDataPath,PTH-paths,"string pointing to a folder for processed data (movement corrected movies, etc). Can be absolute or relative to MotherOfAllFolders (by default, this is the folder containing the YML file)",,str,Processed Data,,
+STG_TempArchivePath,Paths,STG_TempArchivePath,PTH-paths,string pointing to a folder which will be used to archive files generated for the comparison of pipelines. Must be absolute,,str,not set yet,,
+CTV_FeatureNumber,Output,CTV_FeatureNumber,RM-report method,"integer indicating which feature of the CTV to use, where CTV features are numbered 0, 1, 2, 3, ...",,int,0,{flag} >= 0,"Flag {flag_name} can only take integer values 0, 1, 2... Specified value: {flag}"
+CTV_firstframe,Output,CTV_firstframe,RM-report method,"integer indicating a frame number, where frames are numbered 0, 1, 2...",,int,22,,
+CTV_lastframe,Output,CTV_lastframe,RM-report method,"integer indicating a frame number, where frames are numbered 0, 1, 2...",,int,36,,
+CTV_Method,Output,CTV_Method,RM-report method,"specifies curve-to-value function for pixelwise application to movies to generate overviews. To specify a custom-written ctv method, specify function name here and the file containing it in 'CTV_MethodFile'. Or use an internally implemented CTV","22: is the difference between two fixed points
+0: one feature, (definition taken forward from VIEW-IDL) mean of all frames, useful for simulated photographs
+22: one feature, (mean of 3 frames around <last_frame>) – (mean of 3 frames around <first_frame>)
+222: one feature, (mean of 4 frames starting at <last_frame>) – (mean of 4 frames starting at <first_frame>)
+35: one feature, relates to the maximum within 3 secs after stimulus onset
+22and35: two features, of 22 and 35 above
+300: one feature, same as 0
+301: one feature, (definition taken forward from VIEW-IDL) mean of frames 5 to 10, which is generally before stimulus onset. Useful for morphological views.
+302: one feature, mean of frames from <first_frame> to <last_frame> (both inclusive).
+303: one feature, Average of background frames, calculated using first stimulus onset and the flags LE_StartBackground and LE_PrestimEndBackground. Can be useful to visualize and compare baseline values of signals.
+330: one feature, median of all frames
+331: one feature, median of frames 5 to 10, which is generally before stimulus onset. Useful for morphological views.
+332: one feature, median of frames from <first_frame> to <last_frame> (both inclusive).
+333: one feature, Median of background frames, calculated using first stimulus onset and the flags LE_StartBackground and LE_PrestimEndBackground. Can be useful to visualize and compare baseline values of signals.","int, str",22,,
+CTV_MethodFile,Output,CTV_MethodFile,RM-report method,only used when 'CTV_Method',,str,not set yet,,
+CTV_scalebar,Output,CTV_scalebar,RM-report method,"when True, colorbar is added to the right of overviews and movies",,bool,False,,
+CTV_StimulusNumber,Output,CTV_StimulusNumber,RM-report method,"integer indicating which stimulus to use when calculating CTV, where stimuli are numbered 0, 1, 2, 3, ...",,int,0,{flag} >= 0,"Flag {flag_name} can only take integer values 0, 1, 2... Specified value: {flag}"
+CTVM_Method,Output,CTVM_Method,RM-report method,(not yet implemented in pyVIEW)for multiple CTV values at once,,int,0,,
+Signal_FilterSpaceFlag,Filters,FilterSpaceFlag,RM-report method,Toggles on/off the filter applied to sig1 before generating overview frames or movies,,bool,0,,
+Signal_FilterSpaceSize,Filters,FilterSpaceSize,RM-report method,standard deviation of the spatial gaussian filter applied to sig1 before creating overviews/movies. Only used if 'FilterSpaceFlag' is True,,float,3,,
+Signal_FilterTimeFlag,Filters,FilterTimeFlag,RM-report method,Toggles on/off the filter applied to sig1 before generating movies,,bool,0,,
+Signal_FilterTimeSize,Filters,FilterTimeSize,RM-report method,standard deviation of the temporal gaussian filter applied to sig1 before creating movies. Only used if 'FilterTimeFlag' is True,,float,3,,
+RM_differentViews,Output,RM_differentViews,RM-report method,"Change view, e.g. mirror flip right ALs   (SingleOverviews.pro)",,bool,0,,
+RM_Radius,Output,RM_Radius,RM-report method,"integer representing number of pixels, interpreted as the size of glomeruli read from .coor file, used when marking ROIs on overviews and exporting glodatamixes",,int,5,,
+RM_ROIThreshold,Output,RM_ROIThreshold,RM-report method,"When converting spatial footprints of ROIs into binary masks, all pixel above this percentile will be considered to be inside the binary mask",,float,75,,
+RM_ROITrace,Output,RM_ROITrace,RM-report method,Indicates the file from which component masks are sourced,"0 : read the .coor file
+1 : (not tested) read the .tif file (8-bit, value indicates the name of the glomerulus)
+2 : read the .Area file (IDL format).
+3 : read the .roi file created in ILTIS
+4 : read the .roi.tif file created by a processing pipeline
+5 : read the .area.tif file created in ILTIS
+6 : use a fictive ROI with uniform weight over the entire frame",int,3,,
+SO_MV_scalemax,Output,scalemax,RM-report method,value to scale maximum to with SO_indivdiualScale or mv_individualScale equals 0,,float,1,,
+SO_MV_scalemin,Output,scalemin,RM-report method,corresponding value for minimum,,float,0,,
+LE_CalcMethod,CalcSignals,view_CalcMethod,CALC-signal calculation,integer indicating the method used for calculating a biologically relevant signal from raw movie data. Used in view/python_core/calc_methods.py,"0: sig1 = raw1 / 1000
+3: sig1=deltaF/F
+4: sig1=raw1/raw2 - background(raw1/raw2)",int,3,,
+VIEW_batchmode,Xtra,batchmode,SYS-system,"choice between interactive mode and batchmode. Valid values are: True, False ",,bool,True,,
+SO_MV_colortable,Output,colortable,SYS-system,"if integer indicates an IDL_style color table to use. Valid values are: 11-14. Values are 12-14 self-programmed rainbows.
+if string, must be the name of a matplotlib colormap.
+See https://matplotlib.org/tutorials/colors/colormaps.html.
+NOTE: The inverse of every matplotlib colormap is also available, just append '_r' to colormap name. E.g.: Inverse of the colormap 'jet' would be 'jet_r'","
+11: a self programmed colortable
+12: a self-programmed rainbow colortable
+13: a self-programmed rainbow colortable
+14: a self-programmed rainbow colortable
+jet: a matplotlib colormap
+viridis: a matplotlib colormap, etc","int, str",13,,

view/fonts/OpenSans/Apache License.txt

@@ -0,0 +1,201 @@
+Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

view/fonts/PixelOperator/LICENSE.txt

@@ -0,0 +1,121 @@
+Creative Commons Legal Code
+
+CC0 1.0 Universal
+
+    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
+    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
+    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
+    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
+    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
+    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
+    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
+    HEREUNDER.
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without fear
+of later claims of infringement build upon, modify, incorporate in other
+works, reuse and redistribute as freely as possible in any form whatsoever
+and for any purposes, including without limitation commercial purposes.
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific
+works, or to gain reputation or greater distribution for their Work in
+part through the use and efforts of others.
+
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or she
+is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under its
+terms, with knowledge of his or her Copyright and Related Rights in the
+Work and the meaning and intended legal effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not
+limited to, the following:
+
+  i. the right to reproduce, adapt, distribute, perform, display,
+     communicate, and translate a Work;
+ ii. moral rights retained by the original author(s) and/or performer(s);
+iii. publicity and privacy rights pertaining to a person's image or
+     likeness depicted in a Work;
+ iv. rights protecting against unfair competition in regards to a Work,
+     subject to the limitations in paragraph 4(a), below;
+  v. rights protecting the extraction, dissemination, use and reuse of data
+     in a Work;
+ vi. database rights (such as those arising under Directive 96/9/EC of the
+     European Parliament and of the Council of 11 March 1996 on the legal
+     protection of databases, and under any national implementation
+     thereof, including any amended or successor version of such
+     directive); and
+vii. other similar, equivalent or corresponding rights throughout the
+     world based on applicable law or treaty, and any national
+     implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention
+of, applicable law, Affirmer hereby overtly, fully, permanently,
+irrevocably and unconditionally waives, abandons, and surrenders all of
+Affirmer's Copyright and Related Rights and associated claims and causes
+of action, whether now known or unknown (including existing as well as
+future claims and causes of action), in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or
+treaty (including future time extensions), (iii) in any current or future
+medium and for any number of copies, and (iv) for any purpose whatsoever,
+including without limitation commercial, advertising or promotional
+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+member of the public at large and to the detriment of Affirmer's heirs and
+successors, fully intending that such Waiver shall not be subject to
+revocation, rescission, cancellation, termination, or any other legal or
+equitable action to disrupt the quiet enjoyment of the Work by the public
+as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason
+be judged legally invalid or ineffective under applicable law, then the
+Waiver shall be preserved to the maximum extent permitted taking into
+account Affirmer's express Statement of Purpose. In addition, to the
+extent the Waiver is so judged Affirmer hereby grants to each affected
+person a royalty-free, non transferable, non sublicensable, non exclusive,
+irrevocable and unconditional license to exercise Affirmer's Copyright and
+Related Rights in the Work (i) in all territories worldwide, (ii) for the
+maximum duration provided by applicable law or treaty (including future
+time extensions), (iii) in any current or future medium and for any number
+of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the
+"License"). The License shall be deemed effective as of the date CC0 was
+applied by Affirmer to the Work. Should any part of the License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder
+of the License, and in such case Affirmer hereby affirms that he or she
+will not (i) exercise any of his or her remaining Copyright and Related
+Rights in the Work or (ii) assert any associated claims and causes of
+action with respect to the Work, in either case contrary to Affirmer's
+express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+ a. No trademark or patent rights held by Affirmer are waived, abandoned,
+    surrendered, licensed or otherwise affected by this document.
+ b. Affirmer offers the Work as-is and makes no representations or
+    warranties of any kind concerning the Work, express, implied,
+    statutory or otherwise, including without limitation warranties of
+    title, merchantability, fitness for a particular purpose, non
+    infringement, or the absence of latent or other defects, accuracy, or
+    the present or absence of errors, whether or not discoverable, all to
+    the greatest extent permissible under applicable law.
+ c. Affirmer disclaims responsibility for clearing rights of other persons
+    that may apply to the Work or any use thereof, including without
+    limitation any person's Copyright and Related Rights in the Work.
+    Further, Affirmer disclaims responsibility for obtaining any necessary
+    consents, permissions or other rights required for any use of the
+    Work.
+ d. Affirmer understands and acknowledges that Creative Commons is not a
+    party to this document and has no duty or obligation with respect to
+    this CC0 or use of the Work.