View source for Raspberry Pi Timelapse camera project using Python and fswebcam

= About =
First, let me start by saying, '''I love this view!''' It's a million dollar view, to be quite frank, and I do not take it for granted! When I take breaks, I enjoy walking the 7th floor stopping by the Collaboration room for a coffee and to soak up the mountain vista. It's one of the best things about working for Webroot in Broomfield, Colorado.

I am a DevOps engineer for Webroot, and work to automate our Cloud infrastructure...I thought to myself, there's got to be a way to automate this view. 

I specifically wanted to capture sunsets, which my research lead to the Python [https://pythonhosted.org/astral/ Astral] library. This library has methods for querying times of dawn, sunrise, solar noon, sunset, dusk, and more! I moved forward and wrote a proof of concept...
  
Next, I converted my old Android phone into a time-lapse photo taker. I rooted the phone and loaded QPython3. Unfortunately, I discovered limited capability, and QPython3 would mysteriously shut down after some time of running. I needed a more robust platform and stability...

To that end, I purchased a Raspberry Pi 3 and Logitech webcam. I attempted to use Python [https://pypi.python.org/pypi/opencv-python OpenCV] library to control the camera, but it's buggy and I ran into numerous issues. I decided to drop OpenCV for  [https://www.raspberrypi.org/documentation/usage/webcams/ fswebcam] (non-Python), because it's clear cut and reliable. I rewrote my Python script as a wrapper for fswebcam using Python [https://docs.python.org/2/library/subprocess.html Subprocess], which works like a gem. I also use Python [https://pypi.python.org/pypi/APScheduler Apscheduler] to schedule the start of the daily photo shoots based on output from Python Astral.

Please pardon the rudimentary web interface. I am not a web developer. Thus, I chose to use basic frames. I also use Python and a Jinja2 template to generate the time-lapse video web page.

To generate the photo gallery I use [http://www.thregr.org/~wavexx/software/fgallery/ fgallery], because it's minimalistic, fast, and doesn't require server side rendering. It also auto-scales based on your browser size.

I then use a series of hacky Shell and Python scripts to insure the time-lapse process is always running (watchdog - [[Raspberry_Pi_Timelapse_camera_project_using_Python_and_fswebcam#runchecktimelapse|runchecktimelapse]]), transport the newly generated photos to a remote server (rsync), and post-processing once received (copy to proper directories). All of this is governed by cron jobs.

Lastly, to generate the time-lapse video I'm using a shell script which figures out image + next image and creates a transition between the two. Then ffmpeg generates a video for the finished product. This is also governed by a cron job.

The Raspberry Pi internet connection is via Webroot-Guest wifi network. Thus, there shouldn't be a security concern. But, I have also implemented a robust IPTables firewall for good measure, and run SSH on a non-standard port. I also cleared the project with WR-Security.

The power at Webroot sometimes goes out. To protect against this issue, I purchased a power bank which has pass-through - meaning it can charge while delivering power at the same time.

Image captures occur every five minutes. But, because we really care about sunsets, the capture interval is shortened to once every minute between sunset and dusk. 

I hope you enjoy!

P.S.

Some ideas I have kicking around in my head, is a year aggregate time-lapse video where it rolls through each day for 365, or maybe each sunset. The next is using machine learning to train a computer to identify beautiful sunsets or other outliers...I'm working on it.

= Hardware =
Prices as of 2017-12-08
* [https://www.amazon.com/gp/product/B01LWVVMUI/ref=oh_aui_detailpage_o03_s00?ie=UTF8&psc=1 Raspberry Pi 3 Essentials Kit] - $69.99
* [https://www.amazon.com/gp/product/B006JH8T3S/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1 Logitech HD Pro Webcam C920] - $53.99
* [https://www.amazon.com/gp/product/B014RBEAQC/ref=oh_aui_detailpage_o01_s01?ie=UTF8&psc=1 Zendure A3 Portable Charger] - $37.95
Hardware total cost before S/H: '''$161.93'''

= Raspberry Pi =
== Prerequisites ==
<pre>
apt install fswebcam ntpdate screen
</pre>

Python3
<pre>
pip3 install pytz astral apscheduler
</pre>

User and directories.
<pre>
adduser timelapse
mkdir -p /home/timelapse/capture
</pre>

== crontab ==
''/etc/crontab''
<source lang="bash">
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user  command
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#

# Clock updates every three hours
0 0     * * *   root    runclock &> /dev/null
0 6     * * *   root    runclock &> /dev/null
0 12     * * *   root    runclock &> /dev/null
0 18     * * *   root    runclock &> /dev/null


# Daily bounce timelapse for reingest of schedule.
05 0     * * *   root    runtimelapse &> /dev/null

# Check timelapse is running every 15 minutes
*/15     * * * * root    runchecktimelapse &> /dev/null

# Hourly
14 *    * * *   root    runsynctimelapse &> /dev/null
</source>

== Python3 ==
=== timelapse.py ===
''/home/timelapse/timelapse.py''
<source lang="python">
# 2017/12 Michael Grate
# Used to take timelapse pictures. Works as a wrapper for fswebcam.
# Dependencies on pip3 install pytz astral apscheduler
# External dependency on apt install fswebcam

import datetime, subprocess, os
import multiprocessing as mp
from astral import Astral
from apscheduler.schedulers.blocking import BlockingScheduler
from time import sleep


class Timelapse:
    def __init__(self):
        self.sched = BlockingScheduler()
        self.dawnTime = self.getDawnSunsetDusk("DawnTime")
        self.sunsetTime_minus = self.getDawnSunsetDusk("SunsetTime_minus")
        self.duskTime_plus = self.getDawnSunsetDusk("DuskTime_plus")
        self.dawn = self.getDawnSunsetDusk("Dawn")
        self.continualCaptureFileLocation = '/home/timelapse/capture'
        mp.Process(target=self.scheduleJob_day).start()
        mp.Process(target=self.scheduleJob_dawn).start()


    def getTime(self):
        return(int(datetime.datetime.now().strftime("%H%M")))


    def getDateTime(self):
        return(str(datetime.datetime.now().strftime("%Y-%m-%d_%H%M%S")))


    def continuousCaptureImage(self):
        if not os.path.exists(self.continualCaptureFileLocation): # Create the directory if it doesn't exist.
            os.makedirs(self.continualCaptureFileLocation)
        while True:
            if self.getTime() > self.dawnTime and self.getTime() < self.sunsetTime_minus : # If the time is greater than dawn and less than sunset minus some time. Capture an image every 5 mins.
                command = 'fswebcam -i 0 -d v4l2:/dev/video0 -r 1280x720 --fps 10 -S 60 --jpeg 100 --shadow --title "Webroot" --subtitle "Rocky Mountains view West" --info "Broomfield, CO" --save {}/{}-Continuous.jpg'.format(self.continualCaptureFileLocation, self.getDateTime())
                process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
                process.wait()
                sleep(300) # Wait 5 minutes
            elif self.getTime() > self.sunsetTime_minus and self.getTime() < self.duskTime_plus: # Else if the time is greater than sunset minus some time and less than dusk plus. Capture an image every 1 min. Because we love sunsets!
                command = 'fswebcam -i 0 -d v4l2:/dev/video0 -r 1280x720 --fps 10 -S 60 --jpeg 100 --shadow --title "Webroot" --subtitle "Rocky Mountains view West" --info "Broomfield, CO" --save {}/{}-Continuous.jpg'.format(self.continualCaptureFileLocation, self.getDateTime())
                process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
                process.wait()
                sleep(60) # Wait 1 minute
            else:
                sleep(60) # Otherwise wait 1 minute.


    def getDawnSunsetDusk(self, Time):
        # Setup Astral
        city_name = 'Denver'
        a = Astral()
        a.solar_depression = 'civil'
        city = a[city_name]

        # Date fetch
        year = datetime.datetime.today().strftime('%Y')
        month = datetime.datetime.today().strftime('%m')
        day = datetime.datetime.today().strftime('%d')

        sun = city.sun(date=datetime.date(int(year), int(month), int(day)), local=True)

        # Dawn
        if Time == "Dawn": # Used to run apscheduler.
            dawn = str(sun['dawn'])[0:19]
            return(dawn)

        if Time == "DawnTime":
            dawn_hour = str(sun['dawn'])[11:13]
            dawn_minute = str(sun['dawn'])[14:16]
            dawnTime = int('{}{}'.format(dawn_hour, dawn_minute)) # We will use this for range time captures.
            #print(dawnTime) # Debugging
            return(dawnTime)

        # Sunset
        if Time == "SunsetTime_minus":
            sunset_hour = str(sun['sunset'])[11:13]
            sunset_minute = str(sun['sunset'])[14:16]
            sunsetTime = int('{}{}'.format(sunset_hour, sunset_minute)) # We will use this for range time captures.
            sunsetTime_minus = (sunsetTime - 30) # Minus 30 mins.
            #print(sunsetTime_minus) # Debugging
            return(sunsetTime_minus)

        # Dusk
        elif Time == "DuskTime_plus":
            dusk_hour = str(sun['dusk'])[11:13]
            dusk_minute = str(sun['dusk'])[14:16]
            duskTime = int('{}{}'.format(dusk_hour, dusk_minute)) # We will use this for range time captures.
            duskTime_plus = (duskTime + 30) # Plus 30 mins.
            #print(duskTime_plus) # Debugging
            return(duskTime_plus)


    def scheduleJob_day(self):
        # Start continuous image capture if the time is greater than dawn and less than dusk.
        if self.getTime() > self.dawnTime and self.getTime() < self.duskTime_plus:
            self.continuousCaptureImage()


    def scheduleJob_dawn(self):
        # Test
        #test = (datetime.datetime.now())
        #test = str(test + datetime.timedelta(0,10)) # Create test job 5 seconds in the future.
        #self.sched.add_job(self.continuousCaptureImage, 'date', run_date=test)

        # Schedule job at dawn.
        self.sched.add_job(self.continuousCaptureImage, 'date', run_date=self.dawn)
        self.sched.print_jobs()
        self.sched.start()


Timelapse()
</source>

== Shell Scripts ==
=== runclock ===
Keep clock synchronized.

''/usr/sbin/runclock''
<source lang="bash">
#!/bin/bash
 
ntpdate clock.isc.org
</source>

=== runtimelapse ===
Runs the timelapse.py on screen/background. Deletes files older than 2 weeks.

''/usr/sbin/runtimelapse''
<source lang="bash">
#!/bin/bash

# Clean up capture directory files older than 5 days.
find /home/timelapse/capture/ -name "*.jpg" -mtime +5 -exec rm -rf {} \;

# Bounce the timelapse process.
PROCESS_ID=`ps -ef|egrep SCREEN|egrep timelapse_screen|awk '{print $2}'`
kill ${PROCESS_ID}

/usr/bin/screen -dmS timelapse_screen python3 /home/timelapse/timelapse.py
</source>

=== runchecktimelapse ===
Makes sure timelapse.py stays running on screen/background. Works in conjunction with a cron job.

''/usr/sbin/runchecktimelapse''
<source lang="bash">
#!/bin/bash
NAME="timelapse_screen"
PROCESS_ID=`ps -ef|egrep SCREEN|egrep $NAME|awk '{print $2}'`

if [ -z "$PROCESS_ID" ];then
        echo "Process is down. Restarting..."
        /usr/bin/screen -dmS timelapse_screen python3 /home/timelapse/timelapse.py
else
        echo "Process active. No action needed."
fi
</source>

=== runsynctimelapse ===
Synchronization script of images from Raspberry Pi to remote web server. Works in conjunction with a cron job. Not actual SSH port for security purposes. 

''/usr/sbin/runsynctimelapse''
<source lang="bash">
#!/bin/bash
chown -R timelapse:timelapse /home/timelapse
rsync --recursive --progress --verbose --stats --links --times -e "ssh -i /root/.ssh/timelapse -p 222" /home/timelapse/capture/* timelapse@timelapse.fyzix.net:/home/timelapse/continuous/

</source>

= Web Server =
== Prerequisites ==
<pre>
apt install lighttpd fgallery screen python3-pip zip liblcms2-utils ntpdate ffmpeg
</pre>

Python3
<pre>
pip3 install pytz astral jinja2
</pre>

User and directories.
<pre>
adduser timelapse
mkdir -p /home/timelapse/hourly
mkdir -p /home/timelapse/video
mkdir -p /home/timelapse/dawn
mkdir -p /home/timelapse/sunrise
mkdir -p /home/timelapse/noon
mkdir -p /home/timelapse/sunset
mkdir -p /home/timelapse/dusk
</pre>

== crontab ==
''/etc/crontab''
<source lang="bash">
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user  command
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#


# Clock updates every three hours
0 0     * * *   root    runclock &> /dev/null
0 6     * * *   root    runclock &> /dev/null
0 12     * * *   root    runclock &> /dev/null
0 18     * * *   root    runclock &> /dev/null

# Hourly
18 *    * * *   root   cd /home/timelapse && python3 postProcessing.py &> /dev/null
19 *    * * *   root   fgallery -r /home/timelapse/hourly /var/www/hourly &> /dev/null

# Morning
20 6    * * *   root    fgallery -r /home/timelapse/dawn /var/www/dawn &> /dev/null
20 7    * * *   root    fgallery -r /home/timelapse/dawn /var/www/dawn &> /dev/null
20 7    * * *   root    fgallery -r /home/timelapse/sunrise /var/www/sunrise &> /dev/null


# Noon
20 12    * * *   root    fgallery -r /home/timelapse/noon /var/www/noon &> /dev/null
20 13    * * *   root    fgallery -r /home/timelapse/noon /var/www/noon &> /dev/null


# Evening
20 17    * * *   root    fgallery -r /home/timelapse/sunset /var/www/sunset &> /dev/null
20 17    * * *   root    fgallery -r /home/timelapse/dusk /var/www/dusk &> /dev/null
20 18    * * *   root    fgallery -r /home/timelapse/sunset /var/www/sunset &> /dev/null
20 18    * * *   root    fgallery -r /home/timelapse/dusk /var/www/dusk &> /dev/null
20 19    * * *   root    fgallery -r /home/timelapse/sunset /var/www/sunset &> /dev/null
20 19    * * *   root    fgallery -r /home/timelapse/dusk /var/www/dusk &> /dev/null
20 20    * * *   root    fgallery -r /home/timelapse/sunset /var/www/sunset &> /dev/null
20 20    * * *   root    fgallery -r /home/timelapse/dusk /var/www/dusk &> /dev/null
20 21    * * *   root    fgallery -r /home/timelapse/sunset /var/www/sunset &> /dev/null
20 21    * * *   root    fgallery -r /home/timelapse/dusk /var/www/dusk &> /dev/null

# Generate Video
20 21    * * *   root    cd /home/timelapse && makeVideo.sh &> /dev/null
24 21    * * *   root    cd /home/timelapse/video/output && python3 generateVideoHTML.py &> /dev/null

# Cleanup files older than 6 days.
25 21    * * *   root    runcleanup &> /dev/null
</source>

== Python3 ==
=== postProcessing.py ===
Organize files based on their time.

''/home/timelapse/postProcessing.py''
<source lang="python">
# 2017/12 Michael Grate
# Used to post process/organize images.
# Dependencies on pip3 install pytz astral

import datetime, glob, os
from shutil import copyfile
from astral import Astral


class PostProcessing:
    def __init__(self):
        # Setup Astral
        city_name = 'Denver'
        a = Astral()
        a.solar_depression = 'civil'
        city = a[city_name]
        # Date fetch
        year = datetime.datetime.today().strftime('%Y')
        month = datetime.datetime.today().strftime('%m')
        day = datetime.datetime.today().strftime('%d')
        self.sun = city.sun(date=datetime.date(int(year), int(month), int(day)), local=True)
        self.processTest()
        self.processHourly()
        self.processDawn()
        self.processSunrise()
        self.processNoon()
        self.processSunset()
        self.processDusk()


    def getDate(self):
        return(str(datetime.datetime.now().strftime("%Y-%m-%d_")))


    def processTest(self):
        # Test
        testTime = 174939
        # Grab range of files.
        testTime_start = str(testTime - 1000)
        testTime_end = str(testTime + 1000)
        for filename in glob.iglob('./continuous/**', recursive=True):
            identify = filename.replace('./continuous/','').replace(self.getDate(),'').replace('-Continuous.jpg','')
            if testTime_start <= identify <= testTime_end:
                #print(identify) # Debugging
                #print(filename) # Debugging
                newFilename = filename.replace('./continuous/','./test/')
                #print(newFilename) # Debugging
                copyfile(filename, newFilename)


    def processHourly(self):
        # Hourly
        hourTimes = ["60000", "70000", "80000", "90000", "100000", "110000", "120000", "130000", "140000", "110000", "160000", "170000", "180000", "19000", "200000", "210000"]
        # Grab range of files.
        for hour in hourTimes:
            hourTime_start = (str(int(hour) - 1000))
            hourTime_end = (str(int(hour) + 1000))
            # Fix padding.
            if int(hour) >= 60000 and int(hour) <= 90000:
                hourTime_start = ('0' + hourTime_start)
                hourTime_end = ('0' + hourTime_end)
            for filename in glob.iglob('./continuous/**', recursive=True):
                #print(filename)
                identify = filename.replace('./continuous/','').replace(self.getDate(),'').replace('-Continuous.jpg','')
                if hourTime_start <= identify <= hourTime_end:
                    #print(identify) # Debugging
                    #print(filename) # Debugging
                    newFilename = filename.replace('./continuous/','./hourly/')
                    #print(newFilename) # Debugging
                    if os.path.exists(newFilename):
                        pass # Skip the file if it already exists. Otherwise copy it.
                    else:
                        copyfile(filename, newFilename)


    def processDawn(self):
        # Dawn
        dawn_hour = str(self.sun['dawn'])[11:13]
        dawn_minute = str(self.sun['dawn'])[14:16]
        dawn_second = str(self.sun['dawn'])[17:19]
        dawnTime = int('{}{}{}'.format(dawn_hour, dawn_minute, dawn_second)) # We will use this for range time.
        # Grab range of files.
        dawnTime_start = str(dawnTime - 2000)
        dawnTime_end = str(dawnTime + 2000)
        # Fix padding.
        dawnTime_start = ('0' + dawnTime_start)
        dawnTime_end = ('0' + dawnTime_end)
        for filename in glob.iglob('./continuous/**', recursive=True):
            identify = filename.replace('./continuous/','').replace(self.getDate(),'').replace('-Continuous.jpg','')
            if dawnTime_start <= identify <= dawnTime_end:
                #print(identify) # Debugging
                #print(filename) # Debugging
                newFilename = filename.replace('./continuous/','./dawn/')
                #print(newFilename) # Debugging
                if os.path.exists(newFilename):
                    pass # Skip the file if it already exists. Otherwise copy it.
                else:
                    copyfile(filename, newFilename)


    def processSunrise(self):
        # Sunrise
        sunrise_hour = str(self.sun['sunrise'])[11:13]
        sunrise_minute = str(self.sun['sunrise'])[14:16]
        sunrise_second = str(self.sun['sunrise'])[17:19]
        sunriseTime = int('{}{}{}'.format(sunrise_hour, sunrise_minute, sunrise_second)) # We will use this for range time.
        # Grab range of files.
        sunriseTime_start = str(sunriseTime - 2000)
        sunriseTime_end = str(sunriseTime + 2000)
        # Fix padding.
        sunriseTime_start = ('0' + sunriseTime_start)
        sunriseTime_end = ('0' + sunriseTime_end)
        for filename in glob.iglob('./continuous/**', recursive=True):
            identify = filename.replace('./continuous/','').replace(self.getDate(),'').replace('-Continuous.jpg','')
            if sunriseTime_start <= identify <= sunriseTime_end:
                #print(identify) # Debugging
                #print(filename) # Debugging
                newFilename = filename.replace('./continuous/','./sunrise/')
                #print(newFilename) # Debugging
                if os.path.exists(newFilename):
                    pass # Skip the file if it already exists. Otherwise copy it.
                else:
                    copyfile(filename, newFilename)


    def processNoon(self):
        # Noon
        noon_hour = str(self.sun['noon'])[11:13]
        noon_minute = str(self.sun['noon'])[14:16]
        noon_second = str(self.sun['noon'])[17:19]
        noonTime = int('{}{}{}'.format(noon_hour, noon_minute, noon_second)) # We will use this for range time.
        # Grab range of files.
        noonTime_start = str(noonTime - 2000)
        noonTime_end = str(noonTime + 2000)
        for filename in glob.iglob('./continuous/**', recursive=True):
            identify = filename.replace('./continuous/','').replace(self.getDate(),'').replace('-Continuous.jpg','')
            if noonTime_start <= identify <= noonTime_end:
                #print(identify) # Debugging
                #print(filename) # Debugging
                newFilename = filename.replace('./continuous/','./noon/')
                #print(newFilename) # Debugging
                if os.path.exists(newFilename):
                    pass # Skip the file if it already exists. Otherwise copy it.
                else:
                    copyfile(filename, newFilename)


    def processSunset(self):
        # Sunset
        sunset_hour = str(self.sun['sunset'])[11:13]
        sunset_minute = str(self.sun['sunset'])[14:16]
        sunset_second = str(self.sun['sunset'])[17:19]
        sunsetTime = int('{}{}{}'.format(sunset_hour, sunset_minute, sunset_second)) # We will use this for range time.
        # Grab range of files.
        sunsetTime_start = str(sunsetTime - 2000)
        sunsetTime_end = str(sunsetTime + 2000)
        for filename in glob.iglob('./continuous/**', recursive=True):
            identify = filename.replace('./continuous/','').replace(self.getDate(),'').replace('-Continuous.jpg','')
            if sunsetTime_start <= identify <= sunsetTime_end:
                #print(identify) # Debugging
                #print(filename) # Debugging
                newFilename = filename.replace('./continuous/','./sunset/')
                #print(newFilename) # Debugging
                if os.path.exists(newFilename):
                    pass # Skip the file if it already exists. Otherwise copy it.
                else:
                    copyfile(filename, newFilename)


    def processDusk(self):
        # Dusk
        dusk_hour = str(self.sun['dusk'])[11:13]
        dusk_minute = str(self.sun['dusk'])[14:16]
        dusk_second = str(self.sun['dusk'])[17:19]
        duskTime = int('{}{}{}'.format(dusk_hour, dusk_minute, dusk_second)) # We will use this for range time.
        # Grab range of files.
        duskTime_start = str(duskTime - 2000)
        duskTime_end = str(duskTime + 2000)
        for filename in glob.iglob('./continuous/**', recursive=True):
            identify = filename.replace('./continuous/','').replace(self.getDate(),'').replace('-Continuous.jpg','')
            if duskTime_start <= identify <= duskTime_end:
                #print(identify) # Debugging
                #print(filename) # Debugging
                newFilename = filename.replace('./continuous/','./dusk/')
                #print(newFilename) # Debugging
                if os.path.exists(newFilename):
                    pass # Skip the file if it already exists. Otherwise copy it.
                else:
                    copyfile(filename, newFilename)


PostProcessing()
</source>

=== Generate Video HTML from Jinja2 template  ===
==== generateVideoHTML.py ====
Get video names and pass it to Jinja2 template

''/home/timelapse/video/output/generateVideoHTML.py''
<source lang="python">
# 2017/12 Michael Grate
# Generates html based on mp4 files in a directory using Jinja2 templates
# Dependency on pip3 install jinja2

import glob, os, collections
from jinja2 import Environment, FileSystemLoader


class generateVideoHTML:
    def __init__(self):
        self.templatesDir = './templates'
        self.videosDir = './videos'
        self.html_target = './index.html'
        self.html_template = 'index.j2'
        self.generateHTML()            

            
    def getVideoFiles(self):
        file_date_path = collections.OrderedDict() # Use an ordered dictionary to make sure the newest file is organized first. 
        filenames = sorted(glob.iglob('{}/**'.format(self.videosDir)), key=os.path.getctime, reverse=True) 
        #print(filenames) # Debugging
        for filename in filenames:
            #print(filename) # Debugging
            date = filename.replace('./videos/','').replace('_Timelapse.mp4','')
            file_date_path[date] = filename                    
        #print(file_date_path) # Debugging
        return(file_date_path)
    
    
    def generateHTML(self):
        env = Environment(loader=FileSystemLoader(self.templatesDir))
        template = env.get_template(self.html_template)
        template.stream(video_data=self.getVideoFiles()).dump(self.html_target)
        

generateVideoHTML() 
</source>

==== index.j2 ====
Jinja2 template

''/home/timelapse/video/output/templates''
<source lang="python">
<html>
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <style>
                video {
                        width:100%;
                        max-width:1280px;
                        height:auto;
                }
        </style>
</head>
<body>
<font face="arial">
<body bgcolor="#000000" link="#FFFFFF" vlink="#FFFFFF" text="#FFFFFF">
<a href="http://timelapse.fyzix.net/video/videos/">Directory listing</a>
<p>
{% for date, filename in video_data.items() %}
<font size="4"><b>{{ date }}</b></font>
<br>
<video controls preload="none">
        <source src="{{ filename }}" type="video/mp4">
</video>
<br>
<p>&nbsp;&nbsp;&nbsp;<p>
{% endfor %}
</body>
</font>
</html>
</source>

== Shell Scripts ==
=== runclock ===
Keep clock synchronized

''/usr/sbin/runclock''
<source lang="bash">
#!/bin/bash

ntpdate clock.isc.org
</source>

=== runcleanup ===
Cleanup rsync received files older than 6 days.

''/usr/sbin/runcleanup''
<source lang="bash">
#!/bin/bash

find /home/timelapse/continuous/ -name "*.jpg" -mtime +6 -exec rm -r {} \;
</source>

=== makeVideo.sh ===
Generate time-lapse video.

''/home/timelapse/makeVideo.sh''
<source lang="bash">
#!/bin/bash
# 2017/12 Michael Grate
# Genrates time-lapse video from still images.

# Variables
TRANSITION_NUM=8
COUNTER=1
DATE=`date +"%Y-%m-%d"`
#DATE="2017-12-16" # Used for manual date video generation
VIDEO_DIR="/home/timelapse/video"
CAPTURE_DIR="/home/timelapse/continuous"
OUTPUT_DIR="/home/timelapse/video/output/videos"

# Insure directory structure is intact.
mkdir -p ${VIDEO_DIR}/${DATE}
mkdir -p ${VIDEO_DIR}/process
mkdir -p ${OUTPUT_DIR}

# Copy files used for video creation from today's date.
cp ${CAPTURE_DIR}/${DATE}* ${VIDEO_DIR}/${DATE}

cd ${VIDEO_DIR}/${DATE} # Change directory environment.
# Create transitions between images.
files=($(ls -la *.jpg | sort -n -t _ -k 2| awk '{print $9}'))
for (( i=0; i<${#files[@]} ; i+=1 )) ; do
	APPLY_COUNTER=$(printf "%03d" ${COUNTER})
	echo ${files[i]} ${files[i+1]} # Debugging 
	convert ${files[i]} ${files[i+1]} -morph ${TRANSITION_NUM} ../process/${APPLY_COUNTER}.jpg
	((COUNTER=COUNTER+1))
done

# Generate mp4 video.
rm ${OUTPUT_DIR}/${DATE}_Timelapse.mp4 # Nuke the video file if it already exists.
cd ../process/
ffmpeg -pattern_type glob -i '*.jpg' -vcodec libx264 -crf 25 -pix_fmt yuv420p ${OUTPUT_DIR}/${DATE}_Timelapse.mp4
rm -r ./*.jpg
</source>

= Troubleshooting fswebcam GD Error: gd-jpeg: JPEG library reports unrecoverable error =
Ran into an issue where I was getting the following error:
<pre>
verify_jpeg_dht,94: Inserting DHT segment into JPEG frame.
GD Error: gd-jpeg: JPEG library reports unrecoverable error: Not a JPEG file: starts with 0xff 0xd2Captured 61 frames in 0.17 seconds.
</pre>

Could not create pictures larger than '''864x480'''.

To solve this, I set manually configured the Frames per second (fps) using the '''--fps''' flag.

Example Command:
<pre>
fswebcam -v -i 0 -d v4l2:/dev/video0 -r 1280x720 -S 60 --jpeg 100 --save test.jpg
</pre>

= Troubleshooting Timeout waiting for frame =
Error
<pre>
--- Opening v4l2:/dev/video1...
/dev/video1 opened.
--- Capturing frame...
Timed out waiting for frame!
No frames captured.
</pre>

Turn on Verbosity, and identify the number of frames it's capturing. 
<pre>
-v
</pre>

Then try adjusting the ``--fps value``
<pre>
fswebcam -v -i 0 -d v4l2:/dev/video1 -r 1920x1080 --fps 3 --jpeg 100 --shadow --title "Webroot" --subtitle "Rocky Mountains view West" --info "Broomfield, CO" --save blah.jpg
</pre>

= Useful commands =
To discover the capabilities of your webcam use (like resolution) :
<pre>
v4l2-ctl --list-formats-ext
</pre>

= To discover granular features you can use with your webcam  =
* Reference: https://www.mankier.com/1/fswebcam

Depending on the Video Device '''/dev/videoX'''
<pre>
fswebcam -d v4l2:/dev/video0 --list-controls
</pre>

These are specified with 
<pre>
--set or -s
</pre>