Merge remote-tracking branch 'refs/remotes/origin/master'

.gitignore

@@ -0,0 +1,2 @@
+data/i140703-001.nix
+analysis_results.nix

README.md

@@ -1,16 +1,77 @@
+---
+title: "CNS2023 Data Workflows Workshop"
+author: "Michael Denker, Moritz Kern, Thomas Wachtler and Reema Gupta"
+date: 2023-07-15
+---
+
 # Data Workflows Tutorial – CNS 2023, Leipzig
 
-### Brief Description
+This repository contains (to read as will contain) all files required for the CNS*2023 Tutorial titled, "[T08: Using open tools to build efficient workflows for data access, management and analysis](https://cns2023.sched.com/event/1ND5W/t08-using-open-tools-to-build-efficient-workflows-for-data-access-management-and-analysis)". 
+
+
+## Schedule 
+
+The workshop will take place on the 15th of July 2023 in two parts:
+
+- [Part 1 (0900 - 1010 CEST) on Sched](https://cns2023.sched.com/event/1ND5W/t08-using-open-tools-to-build-efficient-workflows-for-data-access-management-and-analysis)
+- [Part 2 (1040 - 1210 CEST) on Sched](https://cns2023.sched.com/event/1NCh7/t08-using-open-tools-to-build-efficient-workflows-for-data-access-management-and-analysis)
+
+### Overview
+
+_Session I: 9:00 -- 10:10 am_
+
+- 9:00-9:10 Welcome, Introduction
+- 9:10-9:30 Introduction to GIN
+- 9:30-9:40 Introduction to the Dataset
+- 9:40-10:10 Primer on Neo I
+
+----- 10:10-10:40 COFFEE BREAK -----
+
+_Session II: 10:40--12:10 am_
+
+- 10:40-11:00 Primer on Neo II
+- 11:00-11:30 Data Analysis with Elephant
+- 11:30-12:10 Data Organization and Storage with NIX
+
+## Requirements
+
+To benefit from the workshop you need to have some experience with the Python programming language. To follow the tutorial, you may use any one of the following three options: 
+
+### 1. Working offline
+
+Before attending the workshop please make sure that either the machine you are working on can run jupyter notebooks and install python packages.
+
+Either download the contents of the [CNS2023-Data-Workflows repository](https://gin.g-node.org/CNS2023-Leipzig/CNS2023-Data-Workflows) via the web or use the command line to clone the repository using `git clone https://gin.g-node.org/CNS2023-Leipzig/CNS2023-Data-Workflows.git`.
+
+To make sure your machine is set up for the workshop, please install the Python requirements running `pip install -r requirements.txt` and start (`jupyter noteboook requirements.ipynb`) and run the requirements jupyter notebook before the workshop. We recommend using [Anaconda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) as a Python virtual environment to make sure you are running the workshop in a clean Python environment.
+
+### 2. EBRAINS Collaboratory
+
+To interactively follow the tutorials online, we suggest creating a free EBRAINS account (https://www.ebrains.eu/page/sign-up) in advance.
+
+### 3. Open Source Brain
+
+TODO
+
+## Dataset Used
+
+### The Reach-2-Grasp experiment
+
+Full data manuscript and dataset
+- Brochier, T., Zehl, L., Hao, Y., Duret, M., Sprenger, J., Denker, M., Grün, S. & Riehle, A. (2018). Massively parallel recordings in macaque motor cortex during an instructed delayed reach-to-grasp task, Scientific Data, 5, 180055. http://doi.org/10.1038/sdata.2018.55
+- https://gin.g-node.org/INT/multielectrode_grasp
+
+## Tutorial Abstract
 
 Neuroscientists today face challenges in managing the growing volume and complexity of data generated through rapid technological and methodological advancements and sophisticated experimental paradigms. Data management tools and methods provide indispensable solutions for researchers to efficiently handle, organize, and analyze datasets, facilitating model validation, refinement, and simulation, while fostering collaborations. This tutorial presents examples combining multiple tools synergistically into a complete digitized workflow, to help researchers manage and control data and analysis processes.
 * odML (https://g-node.org/odml)  is an open, lightweight and flexible format that provides a common schema (with implementations in XML, JSON, YAML) to collect, organize and share metadata in a human- and machine-readable way.
 * NIX (https://g-node.org/nix) is a lean data model and file format for storing fully annotated scientific datasets, i.e. the data together with rich metadata (odML) and their relations in a consistent, comprehensive format. 
-* GIN (https://gin.g-node.org) is a platform for version controlled (git and git-annex) data management and collaboration. It supports any file types and folder structure, provides both web and command-line access, option for local installation, and services including  format validation and data publication (DOI).
+* GIN (https://gin.g-node.org) is a platform for version-controlled (git and git-annex) data management and collaboration. It supports any file type and folder structure, provides both web and command-line access, option for local installation, and services including format validation and data publication (DOI).
 * Neo (http://neuralensemble.org/neo), provides programmatic data objects for working with and representing electrophysiological data, and can read data from many proprietary formats. In combination with NIX, Neo makes electrophysiological data interoperable with generic analysis scripts, tools and services. 
 * Elephant (https://python-elephant.org) provides a large portfolio of standard and advanced methods for analyzing data from neuronal spike trains or time series data, such as LFPs. The Neo data model makes them easily accessible to scientists and applications.
-* Alpaca (https://alpaca-prov.readthedocs.io) enables simple capture of  human-readable provenance of the data processing workflow.
+* Alpaca (https://alpaca-prov.readthedocs.io) enables simple capture of human-readable provenance of the data processing workflow.
 
-#### Background reading:
+### Background reading:
 * Grewe, J., Wachtler, T., Benda, J., 2011. A Bottom-up Approach to Data Annotation in Neurophysiology. Frontiers in Neuroinformatics 5, 16. https://doi.org/10.3389/fninf.2011.00016 
 * Zehl, L., Jaillet, F., Stoewer, A., Grewe, J., Sobolev, A., Wachtler, T., Brochier, T.G., Riehle, A., Denker, M., Grün, S., 2016. Handling Metadata in a Neurophysiology Laboratory. Frontiers in Neuroinformatics 10, 26. https://doi.org/10.3389/fninf.2016.00026 
 * Sprenger, J., Zehl, L., Pick, J., Sonntag, M., Grewe, J., Wachtler, T., Grün, S., Denker, M., 2019. odMLtables: A User-Friendly Approach for Managing Metadata of Neurophysiological Experiments. Front. Neuroinform. 13, 62. https://doi.org/10.3389/fninf.2019.00062 

data/LICENSE.txt

@@ -0,0 +1,5 @@
+Massively parallel multi-electrode recordings of macaque motor cortex during an instructed delayed reach-to-grasp task in the directories datasets and datasets_matlab by Institut de Neurosciences de la Timone (INT), UMR 7289, CNRS – Aix Marseille Université, Marseille, France and Institute of Neuroscience and Medicine (INM-6), Forschungszentrum Jülich, Jülich, Germany is licensed under a Creative Commons Attribution 4.0 International License.
+
+All code in the directories code, code/python-odml, code/python-neo, code/elephant, and code/reachgraspio are each published under the BSD 3 clause licenses.
+
+Please see the LICENSE.txt or LICENSE files in individual subdirectories for detailed respective licenses of code and data.

data/l101210-001_small_cut_60.0s.nix

@@ -0,0 +1 @@
+/annex/objects/MD5-s116142548--fa8b3881398ba6f34236d852bc777356

requirements.txt

@@ -0,0 +1,11 @@
+quantities
+neo>=0.12.0
+elephant[extras]>=0.12.0
+viziphant>=0.2.0
+numpy
+matplotlib
+nixio
+jupyter
+jupyterlab
+RISE
+