How To Host Your Own Data (Only Available On The Enterprise Plan)

Set up your own database and server to store all the user data from your referral program. This means Referral Factory does not see or host any of your sensitive customer data, only you do. One hour setup using Docker 👇

For support during your setup please contact: 

  • Support@referral-factory.com - if you're on the Advanced plan 
  • Your dedicated campaign manager - if you're on the Enterprise plan 

 

Read this before starting:

Follow the steps in this document before creating your first referral campaign and before users share their referral links, otherwise there might be gaps in your data. If you already have active campaigns, and you want to migrate your data to your own server, please contact support before starting. 

 

Time required:

+/- 2 hours if you use our pre-built Docker image, or longer if you opt for the manual process to install. We recommend using our Docker image with an externally hosted persistent database for this setup.

 

Expertise level required:

The Docker setup is fairly simple to follow but does require some understanding of devops, and knowledge of Docker commands. We recommend using our pre-built Docker image when installing as this is the fastest and most secure way to get your own data server up and running. 

 

We do also offer a manual setup, if you would prefer to go that route, but this option is not recommended. Prior experience creating and securing servers and databases on your own server is required for developers that opt to setup your server manually. 

 

Docker notes:

The Docker image is publicly accessible on Docker Hub and can be inspected - the various scripts are stored within the container. We have followed industry standards and utilise the power of Docker’s containerisation to ensure that your data is secure within the container, but securing the host server is not detailed in this guide.

 

Database:

While it is possible to host the data directly within your Docker container, we do not recommend this, and recommend that you setup a persistent database to host the data outside of the container. This will ensure that your data is secure in case of any accidental deletion of your container.

 

Now that we've covered all the housekeeping, let's get started with your installation!

Table Of Contents

  1. How The Data Flows 
  2. Referral Factory Account Requirements (to host your own data)
  3. Setup Guide 
    1. Setting Up The Host Server
    2. Pointing Your Domain
    3. Setting Up The Database Server
    4. Setting Up An Externally Persisted Database

    5. Backing Up Your Database
    6. Preparing Your Environment
    7. Building The Container
    8. Setting Up Your SSL Certificate
  4. Logging Into Your Data Server
  5. Setting Up Your Email Server (and outbound email address)
  6. Adding Teammates
  7. How To Update Your Data Server



1. How The Data Flows 

 

Below you can see how the data flows between Referral Factory and your own server. It's important to note that as long as you host your own database AND connect your own mail server to send user emails, then Referral Factory will never SEE or HOST any of your user data. 

 

Flow diagram showing the way that data will move between your own server and Referral Factory so you can rest assured that your referral program is compliant

2. Referral Factory Account Requirements

  1. To host and secure your own user data, you must be on the Advanced or Enterprise plan. View Referral Factory's pricing plans here. The Advanced and Enterprise plans are specifically designed for larger organisations that have strict security protocols around handling of their customer data. 
  2. You need to have your own domain enabled. After your server is setup, you can connect your domain. Details will be in the setup guide below. 
  3. You need to have your own mail server to connect. If you don't connect your own email server, then your user notifications will be sent using Referral Factory's email server by default, and that would essentially be a data transfer to us. Please do not forget to connect your own email server once your setup is complete! 

 
3. Setup Guide 

 

Below is the step-by-step guide you can follow when installing your own server and database using our pre-built Docker image.

 

Whilst most Linux servers will work for the host server, we recommend using modern Debian-based distributions, such as Ubuntu. We have extensively tested this Docker image on Ubuntu 22.04, for both the host, as well as container environments.

 

In this help guide we will use Digital Ocean  to setup the host server, but you can use any hosting provider you wish such as AWS/GCP/Azure or Linode. Please refer to your chosen hosting provider’s help guides for documentation relating to their services.

 

Let’s get started…

 

A. Setting Up The Host Server 

 

  1. Login to your Digital Ocean account, and tap on “Create” and select “Droplets”
  2. Select a region that meets your legal/proximity requirements.
  3. Under the “Choose an Image” heading, click Marketplace and search for Docker. At the time of writing, it will install Ubuntu 22.04 and Docker 23.0.6, but we have tested up until Docker 24.0.5
  4. Choose your Droplet size. We recommend using the Premium AMD images, with at minimum 2GB of RAM. For campaigns where you expect high volumes, we recommend using bigger plans.
  5. Select your authentication method. We recommend using SSH keys over a password, but the choice is yours. If you choose to use an SSH key and have not yet added an SSH key, you can click on “New SSH Key” and following the steps, or select your required SSH Key(s).
  6. Under “Finalize Details”, you can override the generated hostname to match your own domain (referrals.yourdomain.com) but it is not required.
  7. Click “Create Droplet” and give it a few minutes to complete. Once it is created, it will be visible in the list of your Droplets
  8. Take a note of the IP address for your droplet. You will need it a few times during the process. Replace YOURSERVERIP with this IP address throughout the help guide.
  9. If you choose to use another platform outside of Digital Ocean, or choose a different image on Digital Ocean, you will need to install Docker-Engine once your server has been created. Please see https://docs.docker.com/engine/install/ubuntu/ for installation instructions.

B. Pointing Your Domain

 

You'll now need to update your DNS and add an A-record to point your chosen domain (referrals.yourdomain.com) with the YOURSERVERIP address from step 8 in Setting up your Host Server above.

  1. Choose the domain you'd like to host your referral programs on, this must be a subdomain like referrals.yourdomain.com. 
  2. Navigate to the hosting platform you use to host your website/domain.
  3. Find and click on the Manage button, and from there you should be able to navigate to your DNS settings (this could be called DNS Settings or Manage DNS).
  4. In the DNS Manager section, look for the A records (if filtered), then click Add Record.
  5. Fill in the form fields with the following values:
  6. The record type will be an A-Record.
  7. Host record will be the subdomain you want to publish your campaigns to - this will be a subdomain on your own website. For example: if your custom domain added in Referral Factory is referrals.website.com, the Host Record value must be referrals).
  8. Points To value must be an IP address, and you should use YOURSERVERIP from the server 

  9. TTL can be left as-is or set to a default of 4 hours, or 14400 seconds.
  10. Click Save to add the record.
  11. Log into your Referral Factory account, navigate to Settings > Emails And Domains and save the domain here (for example referrals.yourdomain.com) 

referral lin on own domain - referral factory

 
PLEASE email support@referral-factory.com OR your dedicated campaign manager, to inform us once this is done. We will then authorise the domain so your campaigns route there. 

C. Setting Up The Database Server

 

Throughout this guide we will provide options for both an externally persisted database (recommended) as well as an internally persisted database. While we recommend the externally persisted database, if you choose to use the internal database, the Docker image is setup to persist your MySQL data to a special folder on the host using Docker's Named Volumes. This should ensure that your data is persisted to your container in the event of a restart.

 

Whilst we do make every effort to ensure this data is persisted, we cannot guarantee that it will persist in the event of Docker updates or the host server being modified. Rebuilding the container using the persisted internal data may also cause some unforeseen issues with the build scripts.

 

Supported Database Engines:

  • Internal: MySQL
  • External: MySQL, Postgres, SQLite

D. Setting Up An Externally Persisted Database

 

As with setting up the Host Server above, we recommend using Digital Ocean, as their Managed Database service is great, has additional layers of built in security, but you can use any other service that provides database platforms, like AWS or GCP, or you can install MySQL/Postgres directly into the host server. A tutorial can be found here.

 

If you choose to host the database within the host server, please make sure you run mysql_secure_installation (when using MySQL) during the setup, and then create your database, user and other information. When prompted below for the DB_HOST you would then use YOURSERVERIP. Alternatively, if you choose to use Digital Ocean's Managed Database, you will need to copy the public network connection details.

 

More information on how to setup MySQL databases on Digital Ocean's Managed Database service is available here.

 

Internal Updates: We have provided some scripts which will run on startup (during a build or a restart) as well as provided scripts to update the underlying app within the container (which will need to be run as we make updates to the system). To ensure that we do not need to rebuild the containers over and over, should there be bigger changes, we have also added additional update scripts, which can pull updates (if any) to the internal build scripts, should they be required. You can find the details to update your data server at the bottom of this guide.

 

E. Backing Up Your Database

 

Backups are important, especially if you need to restore your data at any point. However, because of the nature of containerisation and other factors, we cannot automate this process for you. If you are using the recommended externally persisted database, and chose to use Digital Ocean's Managed Database service, you can enable automated backups in your Digital Ocean dashboard. 

 

If you are going to use the internal database, or hosting the database on your host server, you will need to ensure that you are doing, at minimum, daily backups of your database. The less frequent the backups, the larger the gaps in your data.

 

There are a number of ways to do this, but we recommend that you store this data off-server, on something that uses S3 storage, such as Digital Ocean's Spaces,  AWS S3, or GCP Cloud Storage.

 

The steps required would be:

  1. Run a backup of the database to a local file using mysqldump or similar 
  2. Pushing the backup file to remote storage - this can be done via CLI tools - Digital Ocean, AWS and GCP all provide CLI tools that easily integrate with their services
  3. Confirming the backup has been saved remotely, and checking that it is not corrupted.

 

Note: If you are using the internal database, the above steps will need to be done inside the Docker container.

 

F. Preparing Your Environment

 

  1. Open your terminal and log in to your newly created server.
    ssh root@YOURSERVERIP
  2. Open the nano text-editor in your shell. If you’re comfortable in the shell, and prefer to use another editor, please replace the nano command below as necessary. You might need to install them with apt within your host server.
    nano .env
  3. Using one the of templates below, based on your chosen database persistence, replace the values where necessary and paste into your editor and save the file. If you used nano, you can save by pressing Ctrl + X and then pressing Y to save.

    Externally Persisted Database (Recommended)

    # Required
    APP_URL=referrals.yourdomain.com
    SUPERUSER_EMAIL=youremail@domain.tld
    REPO_PULL_TOKEN=<Contact Referral Factory Support for your token>
    USE_EXTERNAL_DB=true
    DB_CONNECTION=mysql|pgsql|sqlite
    DB_HOST=yourhostiporurl
    DB_PORT=3306
    DB_USERNAME=yourusername
    DB_PASSWORD=yourpasswor

    # Optional
    OTP_ENABLED=false # This will disable 2FA - Defaults to true
    DB_DATABASE=yourdbname # Defaults to referral_factory_encryption

    Internal Database

    # Required
    APP_URL=referrals.yourdomain.com
    SUPERUSER_EMAIL=youremail@domain.tld
    REPO_PULL_TOKEN=<Contact Referral Factory Support for your token>
    DB_PASSWORD=supersecretpassword

    # Optional
    OTP_ENABLED=false # This will disable 2FA - Defaults to true
  4. Confirm that the values are saved correctly by typing cat .env into your terminal and pressing enter. It should print out the file with the values you replaced.

G. Building The Container

 

  1. Pull the latest version of the Docker image
    docker pull referralfactory/encryption
  2. Start the Docker container, replacing nameofyourcontainer with something memorable as you will need it a few more times. This will run in a detached mode
    docker pull referralfactory/encryption
  3. Monitor the installation progress by running the following command. Be sure to replace nameofyourcontainer with the name you chose above in step 1.


    If you’re using an external database, you can run the following command.

    docker run -d --env-file .env -p 80:80 -p 9001:9001 -p 443:443 --name 'nameofyourcontainer' referralfactory/encryption:latest

    If you are using the internal database, run the following commands to ensure that the data persists to your host. This will ensure that data is maintained if there are updates to the Docker image.

    docker run -d -v rf_encryption_data:/var/lib/mysql --env-file .env -p 80:80 -p 9001:9001 -p 443:443 --name 'nameofyourcontainer' referralfactory/encryption:latest
  4. Wait for the installation to complete. It should take around 2-3 minutes to finish. Once it is complete, the script will output some information in your terminal with important details about your container. Once you're done, press Ctrl + C to exit the logs.



  5. You can view this information at any time, by running the following code inside your container, or alternatively running the command in the host server, directly through Docker.
# Running from within the container
docker exec -it nameofyourcontainer /bin/bash
cat /var/www/html/docs/container-info.txt

# Running within the host server
docker exec nameofyourcontainer cat /var/www/html/docs/container-info.txt

 

At this point you should be able to login to the app at http://referrals.yourdomain.com, however before you do that we recommend you install an SSL certificate before continuing.

 

H. Setting Up Your SSL certificate

 

Note: If you are using external SSL termination, like Cloudflare or other service providers, you can skip this step and move on to Step 4 below.

 

The Docker container supports using LetsEncrypt SSL out of the box, and is setup to install the SSL certificate directly into the server with very little fuss, including managing the certificate renewals automatically. If you, however, want or need, to use another SSL provider, then you would need to log in to the container, generate the required CSR and modify the internal nginx configuration file: /etc/nginx/sites-available/default

 

The above advanced setup is not explained in this guide, but there are various guides available online, like this, that explain the process.


  1. Login to your container shell to setup your SSL certificate. Remember to replace yourcontainername with the name you chose in the "Building your Container" step above.
    docker exec -it nameofyourcontainer /bin/bash
  2. Install the SSL certificate. You do not need to replace anything here, as it will use the environment variables you created in the .env file.
    sudo certbot --nginx -d $APP_URL -m $SUPERUSER_EMAIL --agree-tos --non-interactive
  3. Once this is completed, it will automatically update the domain configuration in the container, and you should be able to access your newly installed secure server on https://referrals.yourdomain.com.

    4. Logging In To Your Data Server

     

    1. Login to the admin panel by going to https://referrals.yourdomain.com/login and signing in using the $SUPERUSER_EMAIL address you used in the .env file, and the temporary password: password. 

    2. You will be forced at this point to reset your password. 

    3. If you enabled 2FA, it will ask you to setup your 2FA by scanning a QR code, and inputting a verification OTP, and then another OTP to confirm your login. We recommend Google Authenticator or Authy, but other services such as Okta, Microsoft Authenticator will work as well.

    4. Once you've successfully logged in, it will ask you to insert your Referral Factory API key, which you can get from inside your Referral Factory account.  Navigate to Settings > Webhooks And API to generate your API key.

    referral program api



    Please contact support@referral-factory.com or your dedicated campaign manager to confirm once your setup is done, we can then help you test that everything is working as it should. 
     

    5. Setting up your email server and outbound email address 

     

    Because your data is encrypted and hosted on your own server, Referral Factory cannot access this data, or send mails on your behalf. This is done to ensure there is no data transfer from your server to our email server when sending emails to users.

     

    After you've completed the setup, you are going to be required to insert your mail server settings to ensure that mails are sent from your server correctly. There are a number of options available, using platforms such as Mailgun, Postmark, as well as corporate email through Microsoft's Office 365, Google Workspaces, or even using Gmail.

     

    To add your own email server credentials please go to the SETTINGS section in admin portal you just setup on your own server: 

     

     

    6. Adding Teammates  

     

    If you’d like to add teammates to your secure portal where your user data is stored, click on the top right menu and click on "Teammates". Be sure to have added your Mail Settings before doing this, otherwise it will not send out the invites.

    If you enabled 2FA, all teammates will be required to enable 2FA on their first login.

     

    Screenshot 2023-09-12 at 19.03.18

    7. How To Update Your Data Server

     

    You should not need to manually update your data server, as we have setup automated updates that run daily at 00:00 UTC, however, in the event that you need to run updates manually, the steps below outline the process.

     

    1. Open your terminal and log in to your data server.
      ssh root@YOURSERVERIP
    2. Enter your container, by running the following command. Remember to replace nameofyourcontainer with what you chose during the installation.
      docker exec -it nameofyourcontainer /bin/bash
    3. Run the following command which will pull down any changes and database migrations for to the core app.
      /bin/bash /var/www/html/docker-scripts/update-app.sh
    4. You can also combine this, and run it directly from the host server by running:
      docker exec nameofyourcontainer /bin/bash /var/www/html/docker-scripts/update-app.sh
    5. Wait for the updates to complete