View source for Legacy Rivendell 3.6.x on Ubuntu18 04

How to install Rivendell 3.6.8 on Ubuntu 18.04.  It has been reported that these instructions also work on Linux Mint 19.3.

This guide assumes you have the 64 bit build of Ubuntu 18.04 (or a variation such as LUbuntu 18.04) installed and running.  Note that these instructions are based on a variety of sources including the folks at Radiotools.uk, the Raspberry PI build, and notes on the email list.

Note that these instructions do not include support for ASI cards, I do not have an ASI card to test with.

== Install and Update your OS ==
These instructions are for Ubuntu 18.04 and various distributions based on Ubuntu 18.04.  It has been tested with the 64 bit release.  I have not tested with the 32 bit build.

If you are looking for a build of Ubuntu 18.04, you might want to consider LUbuntu. It is a lighter weight distribution that works well.

 http://cdimage.ubuntu.com/lubuntu/releases/18.04.5/release/

I highly suggest using a static IP address for Rivendell.  This can either be set up when installing or later once the OS is installed.

Once installed make sure that Ubuntu is fully updated.  From a shell:
 sudo apt-get -y update
 sudo apt-get -y upgrade

== Install Apache, MySQL, and GIT ==
If you are doing a full stand alone install you'll need MySQL, Apache, and GIT.  If you have a central audio store and / or MySQL server then you may not need Apache and MySQL.
 sudo apt-get -y install apache2
 sudo apt-get -y install mysql-server
 sudo apt-get -y install git

== Set up your users and groups ==

Create a Rivendell group
 sudo addgroup rivendell
 sudo adduser pypad

Add your logged in user to the rivendell group
 sudo usermod -a -G rivendell %username% <-- Substitute your logged in user account

Add your logged in user to the Audio group
 sudo usermod -a -G audio %username% <-- Substitute your logged in user account

If you want this user to be able to access the serial port from Rivendell (useful for GPIO's)
 sudo usermod -a -G dialout %username% <-- Substitute your logged in user account

Log out and log back in so your logged in user picks up all the new groups it is a member of.

== Install Dependencies ==
Note, some of these dependencies may not be needed, but as of writing Rivendell 3.6.7 successfully compiles with these installed.

Note, if it asks about enabling Realtime for Jack, I normally say "yes" but this does not work on all systems.

 sudo apt-get -y install build-essential dpkg-dev fakeroot g++ g++-7 gcc gcc-7 \
 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan4 libatomic1 \
 libaudio-dev libcilkrts5 libcups2-dev libcupsfilters-dev libcupsimage2-dev libdrm-dev libexpat1-dev \
 libfakeroot libfontconfig1-dev libfreetype6-dev libgcc-7-dev libgl1-mesa-dev libgles1 libglu1-mesa-dev \
 libglvnd-core-dev libglvnd-dev libice-dev libiodbc2 libitm1 libjbig-dev libjpeg-dev libjpeg-turbo8-dev \
 libjpeg8-dev liblcms2-dev liblsan0 liblzma-dev libmng-dev libmng2 libmpx2 libmysqlclient20 libopengl0 \
 libpq5 libpthread-stubs0-dev libquadmath0 libsm-dev libstdc++-7-dev libtiff-dev libtiff5-dev libtiffxx5 libtsan0 \
 libubsan0 libx11-dev libx11-doc libx11-xcb-dev libxau-dev libxcb-dri2-0-dev libxcb-dri3-dev libxcb-glx0-dev \
 libxcb-present-dev libxcb-randr0-dev libxcb-render0-dev libxcb-shape0-dev libxcb-sync-dev libxcb-xfixes0-dev \
 libxcb1-dev libxcursor-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxft-dev libxi-dev \
 libxinerama-dev libxmu-dev libxmu-headers libxrandr-dev libxrender-dev libxshmfence-dev libxt-dev libxxf86vm-dev \
 make mesa-common-dev pkg-config x11proto-core-dev x11proto-damage-dev x11proto-dev x11proto-dri2-dev \
 x11proto-fixes-dev x11proto-gl-dev x11proto-input-dev x11proto-randr-dev x11proto-xext-dev \
 x11proto-xf86vidmode-dev x11proto-xinerama-dev xorg-sgml-doctools xtrans-dev libflac++6v5 libid3-3.8.3v5 \
 lame jackd2 screen samba patchage vlc-plugin-jack jackd libcdparanoia-dev libflac++-dev libsamplerate0-dev \
 libid3tag0-dev libid3-3.8.3-dev libcurl4-gnutls-dev libsndfile-dev libpam0g-dev libsoundtouch1-dev \
 libasound2-dev libtwolame-dev libmp3lame-dev libmp4v2-dev libfaad-dev libmad0-dev libjack-jackd2-dev \
 libice-dev libsm-dev libxt-dev libxi-dev libssl-dev build-essential libx11-dev libxext-dev xsltproc evince qt4* \
 libtag1-dev libmysqlclient-dev libqt4-sql-mysql libqt5sql5-mysql qtcreator curl python3-mysqldb python3-pyqt4 \
 python3-pip build-essential libssl-dev libffi-dev python3-dev python3-pyqt4* python3.7 docbook5* autotools-dev \
 libtooli* automake libcoverart1 libcoverart-dev libcoverart-doc libdiscid-dev libdiscid-doc libdiscid0 libmusicbrainz5-2 \
 libmusicbrainz5-dev fop docbook-xsl-ns autofs python3-virtualenv

== Get the Rivendell source from GIT and compile source ==

 wget https://github.com/ElvishArtisan/rivendell/archive/refs/tags/v3.6.8.tar.gz

The current 3.x release is 3.6.8. Expand the source package:

 tar -xfv rivendell-3.6.8.tar.gz

If you want to compile the documentation, set up your Docbooks style sheet location:
 export DOCBOOK_STYLESHEETS=/usr/share/xml/docbook/stylesheet/docbook-xsl-ns/

Change to your Rivendell source folder
 cd rivendell-3.6.8
 ./autogen.sh

If compiling '''with documentation:'''
 ./configure MUSICBRAINZ_LIBS="-L/usr/local/lib -ldiscid -lmusicbrainz5cc -lcoverartcc" --libexecdir=/var/www/rd-bin --sysconfdir=/etc/apache2/conf-available

If compiling '''without documentation:'''
 ./configure MUSICBRAINZ_LIBS="-L/usr/local/lib -ldiscid -lmusicbrainz5cc -lcoverartcc" --libexecdir=/var/www/rd-bin --sysconfdir=/etc/apache2/conf-available --disable-docbook

If configure is successful then compile the source:

 make
 sudo make install
 sudo ldconfig -v
 sudo systemctl daemon-reload
 sudo systemctl enable rivendell.service

With any luck the first "make" will complete.  It will take a long time.  You may see some warnings during the compile, this is normal.

'''Making Pypad Scipts Work'''

Once the compile is complete the setup doesn't always copy the pypad.py file to where it needs to be for pypad scripts to work.  The actual pypad scripts get installed /usr/local/lib64/rivendell/pypad/ but the pypad.py API file needs to be put into /usr/lib/python3/dist-packages/.  Here is how to do this manually.

 sudo cp ./apis/pypad/api/pypad.py /usr/lib/python3/dist-packages/

== Disable PulseAudio ==
This is not always necessary, but sometimes it is required to get audio to work properly.  As root edit the file:
 /etc/pulse/client.conf

Uncomment the autospawn and make it no:
 autospawn = no

Save the configuration file.

== Getting MySQL to work and creating the Rivendell user and database ==
Ubuntu 18.04 by default will only allow a user with elevated root access to log into the root MySQL user account.  In other words you'll have to use sudo to log into the MySQL root account.
 sudo mysql -uroot
 CREATE USER 'rduser'@'%' IDENTIFIED BY 'letmein';
 CREATE DATABASE Rivendell;
 GRANT ALL PRIVILEGES ON Rivendell.* TO 'rduser'@'%' WITH GRANT OPTION;
 FLUSH PRIVILEGES;
 quit;

Of course it is a good idea to change the "letmein" password above to something else.  Just make sure that you remember this password, you will need to put it into your /etc/rd.conf file.

MySQL 5.7 needs Strict mode turned off to avoid errors with Rivendell.  Create the following file:
 sudo nano /etc/mysql/conf.d/disable_strict_mode.cnf

Copy / paste into the file that is created: 
 [mysqld]
 sql_mode=IGNORE_SPACE,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

CTRL-O to write the file, CTRL-X to exit.

If you want others Rivendell workstations to be able to connect to this machine's MySQL across the network then change the Bind in Mysql.

 sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Look for the bind-address= line and update it:
 bind-address            = 0.0.0.0

Restart your MySQL service
 sudo systemctl restart mysql

== Configure Apache ==

Turn on the cgid module:
 sudo a2enmod cgid

Copy the Apache configuration
 sudo cp ~/rivendell/conf/rd-bin.conf /etc/apache2/conf-available/
 sudo ln -s /etc/apache2/conf-available/rd-bin.conf /etc/apache2/conf-enabled/rd-bin.conf
 sudo systemctl restart apache2

== Make /var/snd ==
Create /var/snd where your audio files will get stored
 sudo mkdir /var/snd
 sudo chown username:rivendell /var/snd <-- Note substitute your logged in user name and group`
 sudo chmod ug+rwx /var/snd

== Configure Rivendell ==
Copy the sample configuration file to /etc/rd.conf
 sudo cp ~/rivendell/conf/rd.conf-sample /etc/rd.conf

Edit the file and update with any relevant user names, groups, etc.
 sudo nano /etc/rd.conf

== Create your new database and generate audio ==

If you are setting up a new install, populate your Rivendell database:

 sudo rddbmgr --create --generate-audio

If you are updating from a previous version of Rivendell, then restore and / or update your database with rddbconfig:

 sudo rddbconfig

Note, when updating from a previous 2.x version of Rivendell, I found that I had to start with an empty Rivendell database and use the '''Restore''' option in rddbconfig for the database to properly import and update to the latest schema.

== Configure audio cards for ALSA access ==

If you want to use your audio cards through ALSA, then configure them for access:

 sudo rdalsaconfig

Select the card(s) you want to use and hit Save.

== Start the Rivendell services ==
Use systemd to start the Rivendell services:

 sudo systemctl start rivendell

Then start up rdadmin:
 rdadmin

Check your hosts, audio resources and other settings.  If all looks okay start up rdairplay and see if you can play the test file.

== Optional - Make RDSelect work ==
RDSelect is an application that can be useful if you are running a setup that has more then one server, or where you want to have more then one /var/snd and MySQL server available.  The most common use is when there is a hot standby server that you want workstations to be able to easily switch to in the event of a server failure.  To make RDSelect work on Ubuntu there are a few additional things that need to be done.

For file system mounting, RDSelect uses autofs which by default may not be installed on Ubuntu 18.04.  To install it:

 sudo apt-get -y install autofs

RDSelect requires the Rivendell configuration files to be located in /etc/rivendell.d so the first thing to do is create this folder:

 sudo mkdir /etc/rivendell.d

Next, rd.conf must be located in /etc/rivendell.d and /etc/rd.conf must be a symlink to this file:

 sudo mv /etc/rd.conf /etc/rivendell.d
 sudo ln -s /etc/rivendell.d/rd.conf /etc/rd.conf

Now to run rdselect you will need to run it as root and it needs an additional QT variable passed to work correctly on Ubuntu:

 sudo QT_X11_NO_MITSHM=1 rdselect

You should see the RDSelect window come up and list the available configurations.  You can create additional .conf files in your /etc/rivendell.d folder.  Make sure you have a unique entry for the Label= parameter in each .conf file.  This is the configuration name that you will see when you run rdselect.  If you have rdmonitor running you will see which configuration is running.

== Optional - Disable rdmonitor ==
RDMonitor is a little monitor that sits on your screen to tell you what Rivendell configuration you are using.  This is helpful if you are running a setup with a hot standby server and have configured rdselect to allow you to to easily switch from one server to another.  If you are only running with a single server setup, or are running as a stand alone workstation with MySQL and /var/snd on the same workstation then there is little point in having rdmonitor running.  In this case it just takes up space on the screen.  Here is how to disable it.

The easiest is to just make it no longer executable.  From a shell:

 sudo chmod -x /usr/local/bin/rdmonitor

Then if you reboot you'll find that rdmonitor does not pop up.  If you later decide you want to re-enable it, just make it executable again:

 sudo chmod +x /usr/local/bin/rdmonitor

== Optional - Jack Audio with Promiscuous Mode ==
With Rivendell 3.x.x it is possible to get Rivendell to work with JackAudio in either promiscuous mode or with the older methods outlined below.  In the future with Rivendell 4.x JackAudio using promiscuous mode will be the default and it will not be possible to change the User= field in the rivendell.service file.  As a result the better approach is to use Jack in promiscuous mode as it solves a lot of the issues present in the older methods of using Jack.

What is JackAudio promiscuous mode?  Normally with Jack all Jack-aware clients need to be run under the same user in order to see the Jack server.  With Jack running in promiscuous mode Jack clients can be run under different users and as long as all those users are members of the same group then they can see Jack.  This means that Rivendell can start up the Jack server as root and if any logged in users are part of the audio group then any jack clients that are started will also see the Jack server.  This provides the flexibility of the Approach 2 below with the benefits of the Approach 1 below.  This setup is default when Rivendell 4 is installed but needs a few steps to turn it on in Rivendell 3.

To set this up, create the following file:

 sudo nano /etc/profile.d/rivendell-env.sh

And then copy / paste into the file:
 #
 # Run jackd(1) in promiscuous mode
 #
 export JACK_PROMISCUOUS_SERVER=audio

You also need to add a line to your rivendell.service file:

 sudo nano /lib/systemd/system/rivendell.service

and add the line '''Environment=JACK_PROMISCUOUS_SERVER=audio''' to the '''[Service]''' section of the file:

 [Service]
 LimitNOFILE=4096
 Type=simple
 ExecStart=/usr/local/sbin/rdservice
 PrivateTmp=false
 Restart=always
 RestartSec=2
 StartLimitInterval=120
 StartLimitBurst=50
 Environment=JACK_PROMISCUOUS_SERVER=audio

Save the file, and run a:

 sudo systemctl daemon-reload

Then reboot.  You should be able to set up Jack and have Rivendell start it automatically via the setting in rdadmin --> Manage Hosts --> Jack Settings.  Let Rivendell start Jackd, and then if you start other Jack aware apps in the user-space, they should also see your Jack server.  One issue on Ubuntu is when Rivendell starts Jack it doesn't always allow for connecting directly to the ASLA Jack driver.  The work around is to connect it to the Dummy driver and then use ALSA_OUT or ALSA_IN as described in the Jack wiki article.  See the '''[[Jack]] article''' for additional information on getting Rivendell to start JackAudio.

Note that this method has been tested successfully on both Ubuntu 18.04 and the Raspberry PI running Jack2 with Jackd Version 1.9.9 and higher.

One issue when having Jack start as root via rdadmin, if you later are looking to start Jackd using the logged in user (stopping the Rivendell services first and making sure jackd isn't running) Jack will sometimes throw an error that it can't access a driver.  The issue is that files which Jack creates and are located in /dev/shm end up being owned by root:audio.  To fix this, go to a shell and issue the following command:

 sudo chown username:audio /dev/shm/*

Where username is the logged in user.  Jackd should then start as your logged in user. If you do not have Rivendell set up to start Jack then by restarting the Rivendell services caed daemon should now find jack (running under your locally logged in user) when it starts.

== Optional - Jack Audio (Older Method) ==
Getting Jack to work takes a few extra steps.  Jack and the Rivendell services all need to run under the same user ID in order for them to all see each other.  In the CentOS distribution this is accomplished by having the Rivendell services start up jackd, this can be set up in rdadmin --> Manage hosts.  When set up this way both the Rivendell services and Jack run as the root linux user.  Under Ubuntu there are two approaches you can take for getting Jack to work.

'''A word about users and ALSA.'''  If you have multiple sound cards on your computer and want to have Rivendell use one under ALSA and another under Jack then you will need to use Approach 1 below.  Sound cards set up for ALSA do not show up in Rivendell when the services are started under a user other then Root.

'''Approach 1'''
You can have Rivendell start up Jack under the '''root''' user by following the instructions in the [[Jack]] article.  With these instructions Jack and the Rivendell services will be running under the '''Root''' user the same way that Rivendell is designed to run when running on CentOS.  If you start Jack in this manner it is necessary to have Rivendell start up any Jack clients in its "clients to start" configuration so they will also run under root and also see Jack.  For many this may be the easiest approach to getting things to work with Jack.

'''Approach 2'''
It is possible to start Jack as the logged in user and then start the Rivendell services under this same user.  Using this approach makes it possible to start up additional Jack clients from the desktop and have them see Jack running.  Note that when you use this approach if you have multiple sound cards on your system Rivendell will be unable to access additional sound cards via ALSA.  You '''can''' use these sound cards through Jack with ''ALSA_IN'' and ''ALSA_OUT''.

One disadvantage of this approach is that the services rdcatchd usually does not start properly and rdvairplayd often will start, but it will crash on initial use if you try and use virtual logs.  This only happens when the Rivendell services are started as the locally logged in user, it does not happen when they are run as root.  There is a work around for this (see below).

The process for this approach is to have your system set up for auto-login, have Jack started before the Rivendell services and then have systemd start the Rivendell services under the same logged in user.

First, stop the Rivendell services:
 sudo systemctl stop rivendell

Next remove all audio cards from being selected in rdalsaconfig.  If a sound card is selected in rdalsaconfig then sometimes Rivendell will grab the card instead of Jack.
 sudo rdalsaconfig

To have Systemd start up the Rivendell services under the logged in user, we have to edit the following file:

 sudo nano /lib/systemd/system/rivendell.service

In the [Service] section, add a line '''User=%username%''' where %username% is the logged in user, the same one that you are having start Jack. So if the logged in user is rd, then the [Service] section of the file should look like this:

 [Service]
 Type=simple
 ExecStart=/usr/local/sbin/rdservice
 PrivateTmp=false
 Restart=always
 RestartSec=2
 StartLimitInterval=120
 StartLimitBurst=50
 User=rd   <--- Update the 'rd' with your logged in user - the one you want the daemons and Jack to run under!

**Note, the "U" in User needs to be a capital U or it won't work!

Once that file is updated with the needed user name, save the file and then reload the daemons for systemd (this loads up your changes)
 sudo systemctl daemon-reload

Now start Jack audio.  You can do this with qjackctl or with a command line.  

Once Jack is started, start up your Rivendell services. From a command prompt:

 sudo systemctl start rivendell

you should find that Rivendell now sees Jack.  Jack connections can now be made with Rivendell's Jack Connect [JC] macro or via Patchage or QJackCTL

If this works, then what you can do is disable the Rivendell service from starting up, have the operating system start up Jack with the desktop, and use Cron to start up the Rivendell service.  Be sure that you have your user set up for auto-login (no password required) upon reboot.

Disable Rivendell service from systemd:
 sudo systemctl disable rivendell

Find where your system allows you to add items to its autostart.  In LUbuntu it is found in the 
 Preferences --> Default Applications for LX Session --> Autostart option

Under manually started applications, add in the command to start Jackd. I normally put this into a detached screen session.

Once that is done, modify your root crontab:
 sudo crontab -e

And add in a line for a delay and a start of the Rivendell services:

 @reboot sleep 60 && systemctl start rivendell

If done correctly this will allow Jack to start, then after a brief delay will start up the Rivendell services which will see Jack as it is already running.

**Note, as mentioned it has been found that when running the Rivendell services under the locally logged in user, the service rdcatchd usally does not start and rdvairplayd often crashes the first time you try and use a virtual log.  Both of these services can be started manually and once started manually they seem to work fine.  To check if they are running:

 ps -C rdcatchd -C rdvairplayd

The output should look something like:

  PID TTY          TIME CMD
 1143 ?        00:06:12 rdcatchd
 1225 ?        00:06:57 rdvairplayd

If one of these services is not running, you can start one or both manually from a command shell with:

 rdcatchd &
 rdvairplayd &

==Optional - Get Jack and Pulseaudio to work together ==
This is not really related to Rivendell, but in some situations it is helpful to have Pulseaudio and Jack work together.  This allows you to use audio applications that route their audio through Pulseaudo.  Pulseaudio can be routed through Jack and ultimately Rivendell.  Some users have found this allows them to set up a VOIP softphone application on the Rivendell workstation to allow for remote audio uses such as remote voicetracking.

This approach presumes that you have not previously disabled Pulseaudio and are have already configured Jack for use.  

First you'll need to have the Pulseaudio-Jack bridge module installed:
 sudo apt-get install pulseaudio-module-jack

The process when starting up Jack is that Pulseaudio first needs to be suspended, Jack started, and then the Pulseaudio modules started.  When shutting down the Pulseaudio modules first need to be killed, Jack stopped, and then Pulseaudio started again.  There are a number of ways to accomplish this, but the easiest is to create 4 scripts.  First create a folder for these scripts:

 mkdir ~/scripts/

'''File 1:'''
 sudo nano ~/scripts/1-pulse-jack-prestart.sh

Copy / paste into the file: 
 #!/bin/bash
 pacmd suspend true

'''File 2:'''
 sudo nano ~/scripts/2-pulse-jack-poststart.sh

Copy / paste into the file:
 #!/bin/bash
 pactl load-module module-jack-sink channels=2
 pactl load-module module-jack-source channels=2
 pacmd set-default-sink jack_out
 pacmd set-default-source jack_in

'''File 3:'''
 sudo nano ~/scripts/3-pulse-jack-prestop.sh

Copy / paste into the file:
 #!/bin/bash
 SINKID=$(pactl list | grep -B 1 "Name: module-jack-sink" | grep Module | sed 's/[^0-9]//g')
 SOURCEID=$(pactl list | grep -B 1 "Name: module-jack-source" |grep Module | sed 's/[^0-9]//g')
 pactl unload-module $SINKID
 pactl unload-module $SOURCEID
 sleep 5

'''File 4:'''
 sudo nano ~/scripts/4-pulse-jack-poststop.sh

Copy / paste into the file:
 #!/bin/bash
 pacmd suspend false

Once you have created all these files, make them all executable:
 chmod +x ~/scripts/*

Once you have these scripts created,  if you are using '''Option 2''' from above to start Jack the easiest way to use them is to have qjackctl load them up.  Set them up like this:

[[Image:Jacksetup.png|center|600px|thumb|border|Pulseaudio-Jack Bridge setup]]

Now when you start Jack in qjackctl you should see the Pulseaudio connections in the Connections screen.  You can now connect / route these to any other Jack input / output.

[[File:JackConnections.png | center]]

Now if you restart your Rivendell services you should see the Rivendell connections show up.
 sudo systemctl start rivendell

You can now connect Rivendell audio sources to and from Pulseaudio.

==Optional - Tweak the upper margin of your display ==
I've found that on Lubuntu 18.04 for some reason the top border of some of the windows displays above the top of the screen, making it difficult to move those windows around the desktop.  This is especially bad with '''RDAirplay''' and '''RDAdmin'''.  The way to fix this is to add a slight desktop margin on the top of the screen.  In Lubuntu:

 Click the Menu button --> Preferences --> Openbox Configuration Manager
 On the left menu, click on the Margins tab  and in the "Top" dialog box put 20 so it will read 20 px.
 Click the Close button

Now when you start up rdairplay or rdadmin you'll see the top border of the window and will be able to move it around the screen.  Depending on your screen and resolution you might need more then 20 pixels, try 30 if it is not enough.

== Don't delete your source! ==
A final note about the source that you have just compiled from.  If in the future you are updating to a new version of the source code, you will first need to uninstall the current version.  If you don't do this then the code remains linked to various existing libraries and the newer updated source will give you errors when you try and compile it.  

The easist way to uninstall from source is to go to your original source folder and do a make uninstall:

 cd ~/rivendell-3.6.8/
 sudo systemctl stop rivendell
 sudo make uninstall

If you have used a generic ./rivendell/ folder for your source file name, then once uninstalled I recommend renaming the folder. I usually use reference to the version of the source that the folder contains. As an example:
 cd
 mv ~/rivendell/ ~/rivendell-368/

Once you've done this you can then safely pull down the new source code from github and compile it.