#+title: ApiNATOMY model RDF export and deployment
#+options: num:nil
* ApiNATOMY to RDF/OWL2 conversion
** Basic strategy
   JSON -> RDF -> OWL2 \\
   Conversion from json to rdf should have few if any changes in semantics. \\
   Conversion from a direct rdf translation to OWL2 is where the semantic \\
   translation of ApiNATOMY structures into OWL2 constructs will happen.
* Server setup
On the ontology host (read, ttl host, not SciGraph host) you will need the following.
#+begin_src bash :dir /ssh:host-apinat-ttl|sudo:host-apinat-ttl: :eval never
mkdir /var/www/sparc/ApiANTOMY
mkdir /var/www/sparc/ApiANTOMY/archive
mkdir /var/www/sparc/ApiANTOMY/archive/manual
mkdir /var/www/sparc/ApiANTOMY/ontologies
chown -R nginx:nginx /var/www/sparc/ApiANTOMY
#+end_src
* Export
Make sure SciGraph services and InterLex are accessible for OntTerm.

Run this block in emacs with =C-c C-c= or tangle and run with the block below
# note have to export to working dir not ../bin/ because
# there is no test folder inside of bin and python can't
# look backward up the folder hierarchy to find it
#+name: apinat-export
#+header: :shebang "#!/usr/bin/env bash" :tangle-mode (identity #o0755)
#+begin_src bash :dir ../ :tangle ../export-apinatomy-tests :async
spc apinat ./test/apinatomy/keast-spinal-map.json keast-spinal.ttl
spc apinat ./test/apinatomy/bolser-lewis-map.json test-bolew.ttl
spc apinat keast-spinal-full.jsonld keast-spinal-full.ttl
#+end_src

You can also tangle this file to produce [[file:../export-apinatomy-tests]]
and then run that file from the working directory of this sparc-curation repo.
#+begin_src bash
emacs --batch \
      --load org \
      --load ob-shell \
      --load ob-python \
      --eval '(org-babel-tangle-file "./docs/apinatomy.org")'

./export-apinatomy-tests
#+end_src
* Deploy ttl
First run the export via [[apinat-export][apinat-export]].
Then
#+begin_src bash :results none :noweb yes
scp ~/git/sparc-curation/test-bolew.ttl cassava:/tmp/
scp ~/git/sparc-curation/keast-spinal.ttl cassava:/tmp/
scp ~/git/sparc-curation/keast-spinal-full.ttl cassava:/tmp/
#+end_src

#+begin_src bash :dir /ssh:cassava|sudo:cassava:
DATE=$(date +%Y%m%dT%H%M%S)
ARCHIVE_PATH="archive/manual/${DATE}"
FOLDER="/var/www/sparc/ApiNATOMY/${ARCHIVE_PATH}"
mkdir $FOLDER
mv "/tmp/test-bolew.ttl" $FOLDER
mv "/tmp/keast-spinal.ttl" $FOLDER
mv "/tmp/keast-spinal-full.ttl" $FOLDER
chown -R nginx:nginx $FOLDER
pushd /var/www/sparc/ApiNATOMY/ontologies
unlink test-bolew.ttl
ln -s "../${ARCHIVE_PATH}/test-bolew.ttl"
unlink keast-spinal.ttl
ln -s "../${ARCHIVE_PATH}/keast-spinal.ttl"
unlink keast-spinal-full.ttl
ln -s "../${ARCHIVE_PATH}/keast-spinal-full.ttl"
popd
#+end_src

Check [[https://cassava.ucsd.edu/ApiNATOMY/ontologies/]] for success.
* Load and deploy graph
Then run
[[file:~/git/pyontutils/nifstd/scigraph/README.org::run-load-deploy-graph-sparc-data][run-load-deploy-graph-sparc-data]]
to load and deploy in one shot.

An example run is
#+begin_src bash
~/git/pyontutils/nifstd/scigraph/bin/run-load-graph-sparc-data
~/git/pyontutils/nifstd/scigraph/bin/run-deploy-graph-sparc-data
#+end_src
* Dynamic cypher queries
NOTE: this section contains temporary instructions.
This should really be done on a development instance of data services.
Sometimes it is faster to edit [[tramp:/ssh:aws-scigraph-data-scigraph:services.yaml]] directly.
Use the following command to restart services to load the updated dynamic queries.
#+begin_src bash :results none
ssh aws-scigraph-data sudo systemctl restart scigraph
#+end_src
When you have a query working as desired add it or update it in
[[file:../resources/scigraph/cypher-resources.yaml][cypher resources]].
# TODO need that local/remote git link ...
See also [[file:../../pyontutils/nifstd/scigraph/README.org::#sparc-data-services-build-deploy][data services build and deploy]].
* Add new ApiNATOMY model to SciGraph load
Edit [[file:../resources/scigraph/ontologies-sparc-data.yaml][ontologies-sparc-data.yaml]].
To add a new entry that looks like the following. Change the name =my-model.ttl=
to match the name of the file that you scp to cassava.
#+begin_src yaml
  - url: https://localhost/ApiNATOMY/ontologies/my-model.ttl
    reasonerConfiguration:
      factory: org.semanticweb.elk.owlapi.ElkReasonerFactory
      addDirectInferredEdges: true
      removeUnsatisfiableClasses: true
#+end_src
* ApiNATOMY model server specification
While an ApiNATOMY server has been on the roadmap for some time, there have not been
clear requirements and use cases to drive the development in a way that is productive.
As the conversion of ApiNATOMY models to RDF has progressed, some of the requirements
and use cases have presented themselves and helped to solidify a set of initial use cases.
The need to integrate knowledge represented in ApiNATOMY into the larger linked data space
provides some initial requirements which are the that the server be able to provide persistent
and resolvable identifiers for ApiNATOMY models, and that it be able to provide high granularity
access to the version history of these models. In addition, we are ultimately aiming for
the server to be able to automatically convert input models or spreadsheets into generated
models and resource maps. We have mapped out three phases for arriving at this end goal.
The first phase is to be able to resolve input models, the second is to be able to upload
and link the generated model and resource map and track which input model they came from.
These two will address our primary short-term needs.

To accomplish this, the plan is to use git (via GitHub) as the primary datastore for the models.
This will allow us to leverage the significant existing infrastructure around GitHub for version
control, collaboration, review, content hosting, and backup. In front of this there will be a
server that provides resolvable persistent identifiers for ApiNATOMY models so that the identifiers
appearing in the linked data graphs will be resolvable and interoperable with the rest of the
NIF-Ontology search and discovery tooling.

In the future as part of the third phase we can work towards automating the conversion of input models,
and it might also be possible to have the server automatically convert and serve the RDF version of the
models as well.

A brief outline of the initial requirements needed to meet the needs of the RDF conversion pipeline
are documented below.
** https by default
** url structure
*** apinatomy.org
alternately https://uri.apinatomy.org
**** /uris/models/{model-id}.{ext}
how to deal with json/ttl and model, generated, map
**** /uris/models/{model-id}/ids/{local-id}
**** /uris/readable/{string}
**** /uris/elements/{string}
** transformed models/copies need to be able to point back to the exact commit
for deposition on blackfynn, export to scigraph, etc.
the source model hash needs to be separat
** return authoring metadata
** store the source model
** have endpoint for resource-map and generated
