arrow-left

Only this pageAll pages
gitbookPowered by GitBook
1 of 20

FarmStack

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Quick Start

Deploying Farmstack

Configuring a connector with Google Sheets

Watch this space for super exciting updates. Our bots are already hard at work using GPT-3 to create this page for you.

Guides

Introduction

FarmStack is a reference implementation of an open and interoperable data sharing protocol in agriculture sector.

FarmStack is required because:

  • Relevant farmer profile including farmer activity not available

  • Lack of trust on misuse or under utilisation of data with a centralised data warehouse

  • Need to comply with evolving data policy and privacy safeguarding measures

  • Existing data integration tools lack the customisation

hashtag
Quick start

In browser, open: and follow the instructions

For more details see .

Requirements:

  • Python3.6+

  • Docker Desktop (for MacOS and Windows with wsl2).

  • Docker and Docker Compose for ubuntu

hashtag
Deployment and Guides

You can see FarmStack in action by for fetching data from .

Or you can follow one of our step-by-step guide to setup your own connectors:

hashtag
Features

FarmStack enables network of data providers and consumers through a suite of products and functionalities:

  1. Share data directly without any third party through trusted peer to peer (p2p) connector.

  2. Empower the data provider to restrict usage of data through usage policies.

  3. Give control of data back to the farmers by managing consent using data wallet.

FarmStack is the sum total of all the peer to peer connectors and associated usage policies.

For more information visit or .

hashtag
Contributing

We would love your contribution to this project no matter big or small.

To get started see our .

hashtag
Roadmap

@TODO - Add Roadmap

hashtag
License

Farmstack is licensed under Apache License 2.0. See file for licensing information

Removing farmstack

Dockerization

Enable entities to create plugins to make their data discoverable (description of data).

http://localhost:8000/arrow-up-right
local deployment
Setting up Video Library Connector
Digital Green's Video Libraryarrow-up-right
Configuring a connector with CSV file
Dockerizing applications to be compatible with FarmStack Connector
Install a dockerized application with a connector
our websitearrow-up-right
Confluence Pagearrow-up-right
contribution guide
LICENSEarrow-up-right

Configuring a connector with CSV

hashtag
Overview

This tutorial will describe how to setup FarmStack connector for local csv files. Kindly follow the steps to install FarmStack requirements before proceeding with this setup.

hashtag
Local CSV file

You can follow this process for any file, here we will be using a file called cities.csv present in Downloads directory in home folder.

Clone the FarmStack Github repository on your local machine and open it.

In the FarmStack repository, open example-provide-routes.yaml file in fs-config/usage-control-example/ directory.

In the route sendData, replace sample_data1.csv with filename of your CSV file.

Next, open docker-compose-provider.yaml file in fs-config/usage-control-example/ directory.

Here comment out the lines which mount sample_data1.csv and sample_data2.csv to the docker container, and add the line to mount the cities.csv to the container, as shown here:

There is no limit on the number of CSV files that can be mounted on connector, follow the same instructions for more csv files.

Save the files and in the terminal window type the following command to run FarmStack setup:

After the setup is complete, in the browser window, open the installer frontend by typing .

Follow the steps to create your connectors, give a unique name to your connectors such as cities-provider and cities-consumer.

Start the connection by clicking Setup Connection Button.

When the connector setup is done click View transferred data link to see your data. Kindly wait a couple of minutes for the contract negotiation process of provider and consumer to complete before they can start sharing the data.

hashtag
Online CSV file

Watch this space for super exciting updates. Our bots are already hard at work to create this page for you.

hashtag
Next Steps

This completes the tutorial for CSV file transfer through FarmStack Provider Connector. If you face any issue while setting up your own csv file kindly open a new issue in the github repository and our experts will guide you.

Installing Prerequisites

hashtag
Deployment on Linux, MacOS and Windows (with WSL2)

  • Install Docker on your system according to OS:

git clone https://github.com/digitalgreenorg/farmstack-open.git
cd farmstack-open
python3 setup.py

Configuring a connector with JSON/Rest API

Watch this space for super exciting updates. Our bots are already hard at work using GPT-3 to create this page for you.

Deployment on AWS EC2

Flask

Django

Docker Desktop for MacOS and Windowsarrow-up-right

  • Docker install on Ubuntuarrow-up-right

  • For Ubuntu, also install docker-compose:

    • Install the dependencies on ubuntu:

    sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
    sudo chmod +x /usr/local/bin/docker-compose
    sudo apt install python3-pip python3-venv
    localhost:8000arrow-up-right

    Deploy Farmstack

    hashtag
    Quick deployment on Ubuntu

    • Running this command will install Docker automatically on Ubuntu, if it is not available.

    Contributing to FarmStack

    Thank you for your interest in contributing to FarmStack, currently we are building our contribution guidelines. Meanwhile you can contact us on our and .

    git clone https://github.com/digitalgreenorg/farmstack-open.git
    cd farmstack-open
    code fs-configs/usage-control-example/example-provider-routes.xml
    code fs-configs/usage-control-example/docker-compose-provider.yaml
    python3 setup.py
    websitearrow-up-right
    confluencearrow-up-right
    hashtag
    Deployment on Ubuntu, MacOS and Windows (with WSL2)
    • Install Docker on your system according to OS:

      • Docker Desktop for MacOS and Windowsarrow-up-right.

      • Docker install on Ubuntuarrow-up-right

    • Clone the repository and open it.

    • Run the setup.py file using python3

    After the setup is done, you can view the connector setup UI at: http://localhost:8000arrow-up-right.

    hashtag
    Next steps

    Setting up Self-Managed Connectorschevron-right
    Configuring a connector with CSVchevron-right
    Install a dockerized application with connectorchevron-right
    curl https://raw.githubusercontent.com/digitalgreenorg/farmstack-open/main/scripts/install.sh > fs-install.sh && bash fs-install.sh
    git clone https://github.com/digitalgreenorg/farmstack-open.git
    cd farmstack-open
    python3 setup.py

    Setting up Video Library Connector

    hashtag
    Overview

    This tutorial will guide you through the process of installing and running a Video Library Data Consumer. The video library can be found here: Digital Green Video Libraryarrow-up-right.

    Kindly complete the prerequisites before proceeding with the guide.

    Installing Prerequisiteschevron-right

    hashtag
    Setting up FarmStack connector

    In the terminal window type the following command to run FarmStack setup:

    After the setup is complete, in the browser window, open the installer frontend by typing .

    hashtag
    Running the connector

    Click on the Add New + button, which will open a dialog to setup connector.

    Select the connector you want to run. Here we'll run the Google Sheets connector.

    Click on Next which will take you to the configure tab.

    In the configure tab, enter a google email you would like to share the data with. You can also change the sheet title here.

    Clicking Next will take you to the connect tab.

    In the connect tab, you can see the provider connectors available to connect.

    Select the Video List Provider (DG - Coco) from the list of providers and click Finish.

    On the homepage, the table will update to show the currently running connectors.

    From this table you can open the homepage of the connector or delete the connector by clicking he red bin icon.

    hashtag
    Configuring the connectors

    hashtag
    Google Sheets Connector

    Open the homepage of the App and click on the Sync Data button to fill the data in Google Sheet.

    hashtag
    CSV Connector

    Open the homepage of the App and click on the Sync Data button to generate a csv. Once the CSV file is generated you click on Download option to download the CSV file.

    Setting up Self-Managed Connectors

    hashtag
    Overview

    This tutorial will guide you through the process of running an example self managed connector end-to-end setup. Kindly complete the prerequisites before following this guide.

    hashtag

    Setting up FarmStack connector

    In the terminal window type the following command to run FarmStack setup:

    After the setup is complete, in the browser window, open the installer frontend by typing http://localhost:8000arrow-up-right.

    hashtag
    Running example connectors

    Navigate to New Connector tab

    Give the provider a unique name, for example, Test Provider 1 and click Next

    Give the consumer a unique name, for example, Test Consumer 1 and click Next

    Verify the details and start the connection by clicking Set up Connection button.

    Depending on your system resources and internet speed this step could take anywhere from a few seconds to a few minutes. You can check the progress in the terminal window.

    When the connector setup is complete, open the status tab and click View transferred data link to see your data.

    Kindly wait a couple of minutes for the contract negotiation process of provider and consumer to complete before they can start sharing the data.

    After the negotiation, the provider will start streaming data to the consumer. Kindly refresh to see the data shared into the consumer application.

    This completes the tutorial for Setting up a Managed connector with usage control example. If you face any issue while running the self managed connector, kindly open a new issue in the Github repository and our experts will guide you.

    Installing Prerequisiteschevron-right
    http://localhost:8000arrow-up-right

    Install a dockerized application with connector

    hashtag
    Overview

    This tutorial will describe how to run the dockerized application with the connector. This tutorial does not contain information on dockerization of the app. Kindly dockerization tutorial according to your app.

    You can find the sample-nodejs application used in this tutorial herearrow-up-right.

    hashtag
    Installing dockerized application with connector

    Kindly follow the previous tutorial to locally setup and deploy FarmStack before proceeding with this next step, if not done already.

    Clone the FarmStack repository and open in terminal, using following commands:

    Open prepareConsumerApp.sh in scripts folder in your favorite editor.

    Edit the following variable to according to your application:

    You can also edit the parameters for the example configuration according to your requirements, but it is advised to leave these variables untouched, unless you know what you're doing.

    In the terminal, execute the script from farmstack-open directory:

    This script will create a docker image for your application and modify the required usage control parameters in the example-provider-routes.xml file.

    In the terminal window type the following command to run FarmStack setup:

    After the setup is complete, in the browser window, open the installer frontend by typing .

    Follow the steps to create your connectors, give a unique name to your connectors such as cities-provider and cities-consumer.

    Start the connection by clicking Setup Connection Button.

    When the connector setup is done click View transferred data link to see your data. Kindly wait a couple of minutes for the contract negotiation process of provider and consumer to complete before they can start sharing the data.

    This completes the tutorial for running the dockerized application with the consumer connector. If you face any issue while running your consumer app kindly open a new issue in the github repository and our experts will guide you.

    Local Deployment

    circle-info

    This deployment has been tested on Linux, MacOS and Windows (with wsl2 and Docker Desktop on wsl2).

    hashtag
    Quick deployment on Linux

    • Running this command will install Docker automatically on Linux, if it is not available.

    hashtag
    Deployment on Linux, MacOS and Windows (with WSL2)

    • Install Docker on your system according to OS:

    • Install the dependencies on ubuntu:

    • Clone the repository and open it.

    • Run the setup.py file using python3

    After the setup is done, you can view the connector setup UI at: .

    NodeJS

    hashtag
    Overview

    This tutorial will describe how to convert your NodeJS application to docker application for compatibility with FarmStack connector.

    hashtag

    python3 setup.py
    python3 setup.py
    Installing Prerequisiteschevron-right
    localhost:8000arrow-up-right

    For Ubuntu, also install docker-compose:

    Docker Desktop for MacOS and Windowsarrow-up-right
    Docker install on Ubuntuarrow-up-right
    http://localhost:8000arrow-up-right
    Dockerization Process

    Create a new file named Dockerfile in the application folder and open in your favorite text editor.

    Copy this code into the Dockerfile

    First we are going to define the image we are going to use. Here we are using latest alpine image of node to keep the size of NodeJS application small. You can use any image available in the Node's Dockerhubarrow-up-right.

    Next we will create a directory to copy all our application code inside the image.

    Since we are using node image, node and npm are already installed in this image. We just need to copy our package.json and package-lock.json files.

    Here, we copy package.json files before copying complete project. This is done to take advantage of Docker layers caching and only install dependencies if the files have changed. You can find more information about this herearrow-up-right.

    Now we will copy your app's source code to docker image.

    Our sample-nodejs app binds to port 8081 so we will map this port to docker daemon by using EXPOSE command. If your app uses any other port, kindly change it

    In this last step, define the command to start your application. Our sample application starts withnpm start command. You can also use a shell script file here which executes to start your server.

    This should be your final Dockerfile

    You can find the final Dockerfile herearrow-up-right.

    hashtag
    .dockerignore file

    Create a .dockerignore file in the same directory as your Dockerfile. Add the following lines to the file:

    hashtag
    Next Steps

    Install a dockerized application with connectorchevron-right
    git clone https://github.com/digitalgreenorg/farmstack-open.git
    cd farmstack-open
    # Application parameters
    # This should be a relative path to the config directory or an absolute path.
    CONSUMER_APP_DIR=../../fs-consumer-apps/sample-nodejs
    CONSUMER_APP_NAME='sample-nodejs' #change application name
    CONSUMER_APP_PORT=8081 #Change port according to your application
    SYSTEM_PORT=8081 # Port on the system you want to map to your application's port
    # Config parameters
    CONFIG_DIR='fs-configs/usage-control-example'
    PROVIDER_CONFIG_FILE=example-provider-routes.xml
    CONSUMER_DOCKER_COMPOSE=docker-compose-consumer.yaml
    ./scripts/prepareConsumerApp.sh
    python3 setup.py
    curl https://raw.githubusercontent.com/digitalgreenorg/farmstack-open/main/scripts/install.sh > fs-install.sh && bash fs-install.sh
    sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
    sudo chmod +x /usr/local/bin/docker-compose
    sudo apt install python3-pip python3-venv
    git clone https://github.com/digitalgreenorg/farmstack-open.git
    cd farmstack-open
    python3 setup.py
    touch Dockerfile
    FROM node:alpine
    # Create app directory
    WORKDIR /app
    # Copy package.json and package-lock.json files to image
    COPY package*.json ./
    
    RUN npm install
    # Copy app's source code
    COPY . .
    EXPOSE 8081
    ENTRYPOINT ["npm","start"]
    Dockerfile
    FROM node:alpine
    # Create app directory
    WORKDIR /app
    
    # Copy package.json and package-lock.json files to image
    ADD package*.json ./
    
    RUN npm install
    
    # Copy app's source code
    COPY . .
    
    EXPOSE 8081
    ENTRYPOINT [ "npm", "start" ]
    .dockerignore
    node_modules
    npm-debug.log
    .npm