Преглед на файлове

Make SCRATCH_PATH to default to local scratch/ which we have with .gitignore file

Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Horea Christian преди 9 месеца
родител
ревизия
a390274b8b
променени са 3 файла, в които са добавени 20 реда и са изтрити 5 реда
  1. 4 4
      Makefile
  2. 14 1
      README.md
  3. 2 0
      scratch/.gitignore

+ 4 - 4
Makefile

@@ -13,12 +13,12 @@ IMAGE_TAG=2.0.0-alpha
 FQDN_IMAGE=${REGISTRY}/${REPOSITORY}/${IMAGE_NAME}:${IMAGE_TAG}
 FQDN_LATEX_IMAGE=${REGISTRY}/${REPOSITORY}/${LATEX_IMAGE_NAME}:${LATEX_TAG}
 
-# PATH for scratch directory storing intermidiate results etc; sensible default if unset:
+# PATH for scratch directory storing intermidiate results etc;
+# By default -- all data including intermediate results are under this folder (YODA!)
 ifeq ($(SCRATCH_PATH),)
-	SCRATCH_PATH = /home/$(USER)/.scratch
+	SCRATCH_PATH = $(PWD)/scratch
 endif
 
-
 OCI_BINARY=docker
 SING_BINARY=singularity
 
@@ -87,7 +87,7 @@ analysis-oci:
 		-v ${PWD}/inputs/opfvta_bidsdata:/usr/share/opfvta_bidsdata \
 		-v ${PWD}/inputs/mouse-brain-templates/mouse-brain-templates:/usr/share/mouse_brain_atlases \
 		-v ${PWD}/outputs/:/outputs \
-		-v ${OPFVTA_SCRATCH_DIR}:/root/.scratch \
+		-v ${SCRATCH_PATH}:/root/.scratch \
 		-v ${PWD}/code/:/opt/src/ \
 		${FQDN_IMAGE} \
 		./produce-analysis.sh

+ 14 - 1
README.md

@@ -50,6 +50,19 @@ Run `datalad get .` for each git-annex enabled repository below:
 <!-- `pip install datalad-osf` -->
 <!-- `datalad clone https://finalpushtoOSF -->
 
+## How to re-run
+
+**Note:** if `SCRATCH_PATH` variable is not provided to `make` invocation,
+the folder `scratch/` will contain all intermediate results (around 400GB) but they would
+not be committed to git or git-annex due to `scratch/.gitignore`.  If you do not have
+sufficient space locally, point it somewhere else.
+
+### Everything including paper building
+
+```bash
+make all
+```
+
 ## How to build and use the images
 
 ### Analysis image
@@ -68,7 +81,7 @@ values are what we used). Then the OCI image can be built with:
 The OCI container can be run with the following command.
 We require the scratch directory to be explicitly specified, as it will end up containing large amounts of data (in excess of 200 GB), and any default might accidentally clutter a difficult to locate directory.
 
-`OPFVTA_SCRATCH_DIR="/your/cache/dir" make oci-run`
+`make oci-run`
 
 The container image can be pushed to a container registry:
 

+ 2 - 0
scratch/.gitignore

@@ -0,0 +1,2 @@
+*
+!.gitignore