Publishing/sharing models

Writing your own macsy-model package

The whole package structure and the corresponding files are described in the section Structure of a macsy-model package. It requires five different types of files to be complete:

  • a metadata.yml file (mandatory)

  • a README.md file (mandatory)

  • a LICENSE file (optional but HIGHLY recommended)

  • a model_conf.xml file (optional)

  • macsy-models definition(s) within a definitions folder (mandatory)

  • HMM profiles within a profiles folder (mandatory)

You can create a template for your package by using macsydata init. It will create for you:

  • the data package directory with the right structure.

  • a template of metadata.yaml .

  • a template of README.md file.

  • a generic model_conf.xml file.

  • a LICENSE file if –license option is set.

  • a COPYRIGHT file if –holders option is set.

  • a directory definitions with an example of model definition (model_example.xml to remove before publishing).

  • a directory profiles where to put the hmm profiles corresponding to the models genes.

Sharing your models

If you want to share your models you can create a macsy-model package in your github repository. Several steps are needed to publish your model:

  1. Check the validity of your package with the macsydata check command. You have to run it from within the folder containing your package files. It will report:

    • everything is clear: macsydata displays the next step totake to publish the package

    • warning: it means that the package could be improved.

    It is better to fix it if you can, but you can also proceed to Step 2

    • error: the package is not ready to be published as is. You have to fix the errors before you go to Step 2.

  2. Create a tag, and submit a pull request to the https://github.com/macsy-models organization. This step is very important: without a tag, there is no package. macsydata check only tagged packages. It is also the duty of the model provider to setup a tag with the same name as the version in the metadata.yml file. It is Mandatory to follow a versioning scheme described here:

    If your package is in version 2.0.1 the tag must be 2.0.1. The version or tag must NOT start with letter as v2.0.1 or my_package-2.0.1.

    Warning

    Check that the tag match with the version defined in metadata.yml. To avoid inadvertent mistake place the script below in .git/hooks/ directory. Check that the hook is well named pre-push and it is executable (chmod 755 .git/hooks/pre-push) This script check if you push a tag and if the tag match the version in metadata.yml If it does not match it prevent the push.

    #!/bin/sh

# An example hook script to verify what is about to be pushed.  Called by "git
# push" after it has checked the remote status, but before anything has been
# pushed.  If this script exits with a non-zero status nothing will be pushed.
#
# This hook is called with the following parameters:
#
# $1 -- Name of the remote to which the push is being done
# $2 -- URL to which the push is being done
#
# If pushing without using a named remote those arguments will be equal.
#
# Information about the commits which are being pushed is supplied as lines to
# the standard input in the form:
#
#   <local ref> <local oid> <remote ref> <remote oid>
#
# This script check if you push a tag
# if yes check if the tag match to the version decalred in metadata.yml
# if yes it prevents the push until the tag and the version match
#
# This script is widely inspired from https://gist.github.com/farseerfc/0729c08cd7c82b07000f20105f733b17

remote="$1"
url="$2"

VERSION_FILE="metadata.yml"

tagref=$(grep -Po 'refs/tags/([^ ]*) ' </dev/stdin | head -n1 | cut -c11- | tr -d '[:space:]')


if [[ "$tagref" == ""  ]]; then
    ## pushing without --tags , exit normally
    exit 0
fi

yml_vers=$(grep "vers:" "${VERSION_FILE}" | cut -d ' ' -f 2- | tr -d '[:space:]')

if [[ "$tagref" == "$yml_vers" ]];
then
    ## tag matches ver
    exit 0
else
    Red=$'\e[1;31m'
    Green=$'\e[1;32m'
    Yello=$'\e[1;33m'
    Clear=$'\e[0m'
    echo "${Red}Tag name don't match metadata file. Preventing push.${Clear}"
    echo "${Yello}tag name: $tagref${Clear}"
    echo "${Yello}metadata version: $yml_vers${Clear}"
    echo "${Green}Please fix it:${Clear}"
    echo "${Green}  1. remove tag:${Clear} git tag -d ${tagref}"
    echo "${Green}  2. edit metadata.yml${Clear}"
    echo "${Green}  3. commit metadata.yml:${Clear} git commit -m \"fix metadata vers\" metadata.yml"
    echo "${Green}  4. tag again:${Clear} git tag ${tagref}"
    echo "${Green}  5. and push:${Clear} git push ${remote} ${tagref}"
    exit 1
fi

exit 0

    pre-push .

  3. When your pull request (PR) is accepted, the model package becomes automatically available to the community through the macsydata tool.

If you don’t want to submit a PR you can provide the tag release tarball (tar.gz) as is to your collaborators. This archive will also be usable with the macsydata tool.

Note

macsydata check checks the syntax of the package, but it does not publish anything. It just warns you if something is wrong with the package. Every model provider should check its own package before publishing it. The package publication is done by the git push and the pull request.

Examples of macsydata check outputs:

Your package is syntactically correct:

macsydata check tests/data/models/test_model_package/
Checking 'test_model_package' package structure
Checking 'test_model_package' metadata_path
Checking 'test_model_package' Model definitions
Models Parsing
Definitions are consistent
Checking 'test_model_package' model configuration
There is no model configuration for package test_model_package.
If everyone were like you, I'd be out of business
To push the models in organization:
        cd tests/data/models/test_model_package
Transform the models into a git repository
        git init .
        git add .
        git commit -m 'initial commit'
add a remote repository to host the models
for instance if you want to add the models to 'macsy-models'
        git remote add origin https://github.com/macsy-models/
        git tag 1.0b2
        git push --tags

You received some warnings:

macsydata check tests/data/models/Model_w_conf/
Checking 'Model_w_conf' package structure
Checking 'Model_w_conf' metadata_path
Checking 'Model_w_conf' Model definitions
Models Parsing
Definitions are consistent
Checking 'Model_w_conf' model configuration
The package 'Model_w_conf' have not any LICENSE file. May be you have not right to use it.
The package 'Model_w_conf' have not any README file.
macsydata says: You're only giving me a partial QA payment?
I'll take it this time, but I'm not happy.
I'll be really happy, if you fix warnings above, before to publish these models.

You received some errors:

macsydata check tests/data/models/TFF-SF/
Checking 'TFF-SF' package structure
The package 'TFF-SF' have no 'metadata.yml'.
Please fix issues above, before publishing these models.
ValueError