Wednesday, 29 November 2017

Motion Detector using a PhotoResistor and Arduino

I recently brought two "WeMos D1 WiFi Uno ESP8266 ESP-12E Development Board" hoping to get my first connectivity of two arduino systems via wifi.

Getting the WeMos D1's working

The first task is to set up the arduinos. I could write out how in full, but there's no point in reinventing the wheel, especially when the instructables website does such a good job here:

Once we have this setup its important to understand how we get that connectivity to the internet, what code we need and what the errors and status codes we might see in order to handle them in the code. The best link I have found, or the simplest for Connecting Arduino to Wifi (including status codes) is the one below.

However, the easiest way to get something working straight away is to use: http://www.esp8266learning.com/wemos-webserver-example.php all that is needed is to set your wifi name and password, upload to your Arduino and your done. Follow the IP address that is printed on the serial monitor and you'll have a webpage where you can turn on and off the light of the arduino. 

Try it with any webpage viewable device connected to the same network and voila!

Do this with both Arduino's. One of the Arduino's will be used as a web server and the other will be used to send the sensor information to switch the light on, on the web server in order to "Alert" someone watching the server Arduino that someone has passed the sensor that could be in another room.

Setting up a sensor to output analog (Alert Arduino)

For this we will need;
 - 1 PhotoResistor
 - 3 Male to Male Jumper Cable
 - 1 Resistor
 - 1 Breadboard

We plug the PhotoResistor into the breadboard on one of the centre areas, one leg will be connected to a positive 5V charge from the Arduino. The other will have a resistor across it into a new line and also a male to male jumper cable into one of the analog pins. The final piece is to connect the other end of the resistor to GND (ground).
The sensor will be pointing off the table towards where you want to monitor
We need to add the variable for the photoResistor. 

Before the Setup Code we add:
int sensorPin = A0;    // select the input pin for the potentiometer
int sensorValue = 0;  // variable to store the value coming from the sensor
Within the loop:
sensorValue = analogRead(sensorPin);
Serial.println(sensorValue);

The information for this was taken from here: https://www.arduino.cc/en/Tutorial/AnalogInput
Once this runs we will start to see the unhindered sensor reading. In my case, this was around 355 - 380 and when covered by hand close and far the readings varied from 150 - 350. 

We can then add some code to the loop to turn the light on/off depending on the value returned by the sensor.
   if(sensorValue > 350){
      //turn light off
   }else{
      //turn light on
   }

Waiting for the Alert (Server Arduino)

We need to add the listener to turn on the light when the sensor is triggered.

In the "match request" section underneath the /LED we need to setup actions when the sensors alert.
  if (request.indexOf("/sensor=ON") != -1){
    digitalWrite(ledPin, HIGH);
    value = LOW;
  }
  if (request.indexOf("/sensor=OFF") != -1){
    digitalWrite(ledPin, LOW);
    value = LOW;
  }
Upload.

To test we can go to 
IPAddress:/sensor=ON
and then
 IPAddress:/sensor=OFF
on a network connected device. We should see the server Arduino's light turn on and then off. 

Posting an Alert when the sensor is triggered (Alert Arduino)

After a little google, the best Arduino POST example I could find using the examples we have was the answer by "stiff" on the following stack overflow question: https://stackoverflow.com/questions/3677400/making-a-http-post-request-using-arduino

Where we need to add;
   #include <ESP8266HTTPClient.h> 
   //inside the loop
   HTTPClient http;    //Declare object of class HTTPClient
   http.begin("http://IPAddress/sensor=OFF");      //Specify request destination
   http.addHeader("Content-Type", "text/plain");  //Specify content-type header
   int httpCode = http.POST("Message from ESP8266");   //Send the request
   String payload = http.getString();                  //Get the response payload

   Serial.println(httpCode);   //Print HTTP return code
   Serial.println(payload);    //Print request response payload
   http.end();  //Close connection
We can then use the:
 http.begin("http://IPAddress/sensor=ON");
with the sensorValue IF statement we made earlier.
if(sensorValue > 350){
   http.begin("http://IPAddress/sensor=OFF");      //Specify request destination
 }else{
   http.begin("http://IPAddress/sensor=ON");
  }
We have now completed the minimum viable product. I will share the full source code below as well as a video of the result. If you have any questions or would like the source code directly feel free to comment!
*UPDATE: below the original source code is an adapted piece of code which is much quicker and more responsive! enjoy!

Full Source Code: 

Alert Arduino: (With Sensor)

#include <ESP8266HTTPClient.h>
#include <ESP8266WiFi.h>

const char* ssid = "XXX";
const char* password = "YYY";

int ledPin = D5;
int sensorPin = A0;
int sensorValue = 0;

void setup() {

  Serial.begin(115200);                 //Serial connection
 
  // Connect to WiFi network
  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);   //WiFi connection

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");

}

void loop() {
   sensorValue = analogRead(sensorPin);
   Serial.println(sensorValue);
   if(WiFi.status()== WL_CONNECTED){   //Check WiFi connection status

   HTTPClient http;    //Declare object of class HTTPClient
   if(sensorValue > 350){
   http.begin("http://
IPAddress/sensor=OFF");      //Specify request destination
   }else{
   http.begin("http://IPAddress/sensor=ON");
   }
   http.addHeader("Content-Type", "text/plain");  //Specify content-type header
   int httpCode = http.POST("Message from ESP8266");   //Send the request
   String payload = http.getString();                  //Get the response payload

   Serial.println(httpCode);   //Print HTTP return code
   Serial.println(payload);    //Print request response payload
   http.end();  //Close connection

 }else{

    Serial.println("Error in WiFi connection");

 }

  delay(10);

Server Arduino:

#include <ESP8266WiFi.h>

const char* ssid = "XXX";
const char* password = "YYY";

int ledPin = D5;
WiFiServer server(80);

void setup() {
  Serial.begin(115200);
  delay(10);


  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, LOW);

  // Connect to WiFi network
  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");

  // Start the server
  server.begin();
  Serial.println("Server started");

  // Print the IP address
  Serial.print("Use this URL : ");
  Serial.print("http://");
  Serial.print(WiFi.localIP());
  Serial.println("/");

}

void loop() {
  // Check if a client has connected
  WiFiClient client = server.available();
  if (!client) {
    return;
  }

  // Wait until the client sends some data
  Serial.println("new client");
  while(!client.available()){
    delay(1);
  }

  // Read the first line of the request
  String request = client.readStringUntil('\r');
  Serial.println(request);
  client.flush();

  // Match the request

  int value = LOW;
  if (request.indexOf("/LED=ON") != -1) {
    digitalWrite(ledPin, HIGH);
    value = HIGH;
  }
  if (request.indexOf("/LED=OFF") != -1){
    digitalWrite(ledPin, LOW);
    value = LOW;
  }
  if (request.indexOf("/sensor=ON") != -1){
    digitalWrite(ledPin, HIGH);
    value = LOW;
  }
  if (request.indexOf("/sensor=OFF") != -1){
    digitalWrite(ledPin, LOW);
    value = LOW;
  }



  // Return the response
  client.println("HTTP/1.1 200 OK");
  client.println("Content-Type: text/html");
  client.println(""); //  do not forget this one
  client.println("<!DOCTYPE HTML>");
  client.println("<html>");

  client.print("Led pin is now: ");

  if(value == HIGH) {
    client.print("On");
  } else {
    client.print("Off");
  }
  client.println("<br><br>");
  client.println("Click <a href=\"/LED=ON\">here</a> turn the LED on pin 5 ON<br>");
  client.println("Click <a href=\"/LED=OFF\">here</a> turn the LED on pin 5 OFF<br>");
  client.println("</html>");

  delay(1);
  Serial.println("Client disconnected");
  Serial.println("");

}
Results:


UPDATE: From the above video we can see that the response is pretty slow with the code we have. To fix this I have edited the "Alert Arduino" code. Below is the results of changes made (below that will be the code).


Video Links: https://www.youtube.com/watch?v=Pfqs-WN9OiQ
                      https://www.youtube.com/watch?v=iTQJmba_wcw

Updated Code:

Alert Arduino: (With Sensor)
#include <ESP8266HTTPClient.h>
#include <ESP8266WiFi.h>

const char* ssid = "XXX";
const char* password = "YYY";

int ledPin = D5;
int sensorPin = A0;
int sensorValue = 0;
int sensorOn = 0;
int currentSensor = 0;

void setup() {

  Serial.begin(115200);                 //Serial connection
 
  // Connect to WiFi network
  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);   //WiFi connection

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");

}

void loop() {
   sensorValue = analogRead(sensorPin);
   Serial.println(sensorValue);
   if(sensorValue > 350){
    sensorOn = 0;
   }else{
    sensorOn = 1;
   }
 
   if(WiFi.status()== WL_CONNECTED){   //Check WiFi connection status

   HTTPClient http;    //Declare object of class HTTPClient
 
   if(currentSensor == 1 && sensorOn == 0){
 
    currentSensor = 0;
    http.begin("http://IPAddress/sensor=OFF");      //Specify request destination
    http.addHeader("Content-Type", "text/plain");  //Specify content-type header
    int httpCode = http.POST("Message from ESP8266");   //Send the request
    http.end();  //Close connection
 
   }else if(currentSensor == 0 && sensorOn == 1){
 
    currentSensor = 1;
    http.begin("http://IPAddress/sensor=ON");
    http.addHeader("Content-Type", "text/plain");  //Specify content-type header
    int httpCode = http.POST("Message from ESP8266");   //Send the request
    http.end();  //Close connection
 
   }else{
 
    //do nothing
 
   }
   //String payload = http.getString();                  //Get the response payload
   //Serial.println(httpCode);   //Print HTTP return code
   //Serial.println(payload);    //Print request response payload

 }else{

    Serial.println("Error in WiFi connection");

 }

  delay(500);

}

You don't need a PHD to open a .phd file!

Today I was looking at some Java Core dumps on a system and wanted to start debugging to see what was causing an apparent memory leak. All was going well until I was led to:
  /log/javacore-filename.phd
I hadn't stumbled across a Portable Heap Dump (phd) before, so I started to have a little google as to how I could open it but couldn't find a command line answer to my problem. Apparently no conversion will get me what I want and I'll have to use a set of tools.

One of the first things that showed up was "IBM Memory Analyzer*" which gives analysis on heap/core dumps. To install you can have a stand alone application but it more conveniently comes as a plugin to be installed directly into an Eclipse runtime.
    http://www.eclipse.org/downloads/eclipse-packages/ 

*Being an IBMer myself I was inclined to use this since its one of our own products.*

To install you simply open eclipse and go to: help -> eclipse marketplace -> search for: "memory analyser" and select go -> install -> follow instructions and restart eclipse

I thought, brilliant I should be able to load the .phd and get the data I need. However, when I tried to load my file I got:
Error opening heap dump 'javaheapdump-user-cmc.phd'. Check the error log for further details.
Error opening heap dump 'javaheapdump-user-cmc.phd'. Check the error log for further details.
Not a HPROF heap dump (java.io.IOException)
Not a HPROF heap dump
After a little more google investigation I found I needed a little extra component called "IBM Monitoring and Diagnostic Tools for Java" or "IBM DTFJ" for short. This works on top of memory analyser and will allow us to open the phd file.

To download it go to: help -> Install new software -> Select the add button next to the "work with" text bar. -> In both boxes type: http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/runtimes/tools/dtfj/ -> toggle "IBM Monitoring and Diagnostic Tools" -> install -> follow instructions and reset eclipse.

Now when you go to open the .phd file, memory analyser will be able to find it and will use DTFJ to open it. From here we can get component reports and leak suspects.

Wednesday, 22 November 2017

Simulating Football / Soccer Results

I've recently started to look for a football simulation engine (soccer over in the USA) that can be used  to begin a football manager game. I realised quickly this is something lots of people try but don't necessarily finish or keep on top of:
- https://github.com/d-nation/soccer-sim-engine
https://github.com/atas76/SimpleFootie
and there's plenty of online forums asking how you might make a simple match simulator, so I thought why not just give it a go and see what I come up with.

My code language of choice is Javascript and I'll be using NodeJS throughout.

Attempt 1 - Use Team Rating to generate prediction

This uses a function that generates a random number
function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}
we can then feed minimum and maximum numbers to get a random number between the two.
This attempt uses the team rating to determine different maximums, so that better rated teams can get a higher score but it isn't impossible for the lower teams to win.
function getHighestLikely(teamRating) {
    if (teamRating < 100 && teamRating > 80) {
        return 20;
    } else if (teamRating < 79 && teamRating > 60) {
        return 8;
    } else if (teamRating < 59 && teamRating > 30) {
        return 2;
    } else if (teamRating < 29 && teamRating > 0) {
        return 1;
    }
}
we can then run the following where the team ratings can be passed in by the user:
var team1rating = 73;
var team2rating = 45;
var homeTeamScore = getRandomNumber(0, getHighestLikely(team1rating)); 
var awayTeamScore = getRandomNumber(0, getHighestLikely(team2rating));
console.log("Home: " + homeTeamScore + " - " + awayTeamScore + " : Away");

Attempt 2 - Random Event, Random Team Member and Player Rating Comparison

I then looked at how we can utilise some random shooting events alongside a random team member and a comparison of two players ratings.

For this we will need:
- an events variable for goal scoring events
- readFile function (will show below)
- two JSON files with players and player ratings
function readFile(filePath) {
    return new Promise(function (resolve, reject) {
        fs.readFile(filePath, 'utf8', function (err, data) {
            if (err) {
                reject(err);
            } else {
                data = JSON.parse(data);
                resolve(data);
            }
        })
    });
}
We use the above function to read in two json files with player ratings, in this example I've put a lot of information that I may or may not use in the future:
{
  "name": "Brewers",
  "rating": "89",
  "players": [{
      "name": "Bill Gallagher",
      "position": "GK",
      "rating": "75",
      "skill": {
        "passing": "78",
        "shooting": "12",
        "saving": "75",
        "penalty_taking": "43"
      }
    },
    {
      "name": "Fred Gallagher",
      "position": "LB",
      "rating": "90",
      "skill": {
        "passing": "83",
        "shooting": "40",
        "tackling": "32",
        "penalty_taking": "53"
      }
    },
    {
      "name": "George Gallagher",
      "position": "CB",
      "rating": "84",
      "skill": {
        "passing": "78",
        "shooting": "37",
        "tackling": "21",
        "penalty_taking": "66"
      }
    },
    {
      "name": "Jim Gallagher",
      "position": "CB",
      "rating": "75",
      "skill": {
        "passing": "33",
        "shooting": "76",
        "tackling": "76",
        "penalty_taking": "21"
      }
    },
    {
      "name": "Sid Gallagher",
      "position": "RB",
      "rating": "82",
      "skill": {
        "passing": "66",
        "shooting": "65",
        "tackling": "81",
        "penalty_taking": "88"
      }
    },
    {
      "name": "Gregory Gallagher",
      "position": "LM",
      "rating": "87",
      "skill": {
        "passing": "51",
        "shooting": "88",
        "tackling": "81",
        "penalty_taking": "94"
      }
    },
    {
      "name": "Arthur Gallagher",
      "position": "CM",
      "rating": "41",
      "skill": {
        "passing": "33",
        "shooting": "66",
        "tackling": "55",
        "penalty_taking": "1"
      }
    },
    {
      "name": "Cameron Gallagher",
      "position": "CM",
      "rating": "99",
      "skill": {
        "passing": "88",
        "shooting": "95",
        "tackling": "91",
        "penalty_taking": "62"
      }
    },
    {
      "name": "Tanisha Gallagher",
      "position": "RM",
      "rating": "79",
      "skill": {
        "passing": "56",
        "shooting": "79",
        "tackling": "74",
        "penalty_taking": "32"
      }
    },
    {
      "name": "Aiden Gallagher",
      "position": "ST",
      "rating": "75",
      "skill": {
        "passing": "83",
        "shooting": "88",
        "tackling": "59",
        "penalty_taking": "45"
      }
    },
    {
      "name": "Louise Peverley",
      "position": "ST",
      "rating": "88",
      "skill": {
        "passing": "73",
        "shooting": "61",
        "tackling": "44",
        "penalty_taking": "66"
      }
    }
  ],
  "manager": "Aiden",
  "formation": [4,4,2]
}
the two json files differed only slightly.
I will also require a variable called "events" : var events = ["shot", "penalty"];
function playMatch(team1Config, team2Config) {
    var matchEventsNo = getRandomNumber(0, 10);
    console.log("Total Events in the Match: " + matchEventsNo);
    readFile(team1Config).then(function (team1) {
        readFile(team2Config).then(function (team2) {
            while (matchEventsNo != 0) {
                console.log("Event: " + matchEventsNo);
                var eventTeam = getRandomNumber(1, 2);
                var eventPlayerHome = team1.players[getRandomNumber(0, 10)];
                var eventPlayerAway = team2.players[getRandomNumber(0, 10)];
                if (eventTeam === 1) {
                    console.log("Event Team: " + team1.name);
                } else if (eventTeam === 2) {
                    console.log("Event Team: " + team2.name);
                }
                var thisEvent = events[getRandomNumber(0, 1)];
                console.log("Event Type: " + thisEvent);
                if (thisEvent === "penalty") {
                    if (eventTeam === 1) {
                        console.log("Player: " + eventPlayerHome.name + "(" + eventPlayerHome.position + ") rating: " + eventPlayerHome.rating);
                        console.log("Player: " + team2.players[0].name + "(" + team2.players[0].position + ") rating: " + team2.players[0].rating);
                        if (eventPlayerHome.rating > team2.players[0].rating) {
                            homeScore++;
                        }
                    } else if (eventTeam === 2) {
                        console.log("Player: " + eventPlayerAway.name + "(" + eventPlayerAway.position + ") rating: " + eventPlayerAway.rating);
                        console.log("Player: " + team1.players[0].name + "(" + team1.players[0].position + ") rating: " + team1.players[0].rating);
                        if (eventPlayerAway.rating > team1.players[0].rating) {
                            awayScore++;
                        }
                    }
                } else if (thisEvent === "shot") {
                    if (eventTeam === 1) {
                        console.log("Player: " + eventPlayerHome.name + "(" + eventPlayerHome.position + ") rating: " + eventPlayerHome.rating);
                        console.log("Player: " + team2.players[0].name + "(" + team2.players[0].position + ") rating: " + team2.players[0].rating);
                        if (eventPlayerHome.rating > team2.players[0].rating) {
                            homeScore++;
                        }
                    } else if (eventTeam === 2) {
                        console.log("Player: " + eventPlayerAway.name + "(" + eventPlayerAway.position + ") rating: " + eventPlayerAway.rating);
                        console.log("Player: " + team1.players[0].name + "(" + team1.players[0].position + ") rating: " + team1.players[0].rating);
                        if (eventPlayerAway.rating > team1.players[0].rating) {
                            awayScore++;
                        }
                    }
                }
                matchEventsNo--;
                console.log(team1.name + " " + homeScore + " - " + awayScore + " " + team2.name);
            }
        });
    });
}

function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}
This created up to ten random events per "Match". For each match it would then decide which of two events had occurred. Either a penalty or a shot. It would then decide which of the two teams (randomly again) has incurred the event. We then channel a series of if statements to determine if there was a goal by comparing the player who took the shot/penalty (randomly chosen from the team) against the opposition goalkeepers ability.

We can run this using:
playMatch("teams/team1.json", "teams/team2.json").then(function(){ 
});

This gives us a limited number of goals (10 max) but can be increased by saying a minimum of X events and a maximum of (infinite) events to increase goals that can be scored.

Random "Plays" made of up to 10 random events.

Whilst this was good and worked, I thought the best way to boost this method of generating football events and therefore goals was to create "plays" which would consist of 10 random events. These would begin by an event starter such as a throw in, corner, goal kick, free kick or a penalty.

All of these are events that "start" play from a stoppage. After this has been determined a continue of the event is decided as either pass, cross or shoot.  A play is only finished being generate once a "shoot" token has been given.

Once the 10 plays have been decided with their potentially infinite events, there is a formula run to see if a goal is scored. First we decide which team is doing each of the 10 events (an array of 10, each number in the array either being 0 or 1 to determine if the home or away team are performing the "play").

We then compare the players skill rating for the given event against a random number between 0 and 100. Thus higher rated players are more likely to be successful in whatever event they are doing i.e. passing. At any stage a pass, cross, shot or penalty can miss thus ending the play and meaning no goal will be scored.

var fs = require("fs");
var async = require("async");
var teamName;
var userName;
var eventStart = ["throwin", "corner", "goalkick", "freekick", "penalty"];
var events = ["pass", "cross", "shoot"];
var homeScore = 0;
var awayScore = 0;
var play = [];
var eventTeams = [];

playMatch("teams/team1.json", "teams/team2.json");
resetVars();

function playMatch(team1Config, team2Config) {
    eventTeam().then(function () {
        readFile(team1Config).then(function (team1) {
            readFile(team2Config).then(function (team2) {
                async.eachSeries(eventTeams, function eachTeam(thisTeam, thisTeamCallback) {
                    console.log("--------------------");
                    console.log("Starting Play");
                    if (thisTeam === 0) {
                        generatePlay().then(function () {
                            goalScored(team1, team2, team1).then(function (score) {
                                if (score === 1) {
                                    console.log("Team 1 scored");
                                    homeScore++;
                                    play = [];
                                    thisTeamCallback();
                                } else {
                                    console.log("Team 1 missed");
                                    play = [];
                                    thisTeamCallback();
                                }
                            });
                        });
                    } else {
                        generatePlay().then(function () {
                            goalScored(team1, team2, team2).then(function (score) {
                                if (score === 1) {
                                    console.log("Team 2 scored");
                                    awayScore++;
                                    play = [];
                                    thisTeamCallback();
                                } else {
                                    console.log("Team 2 missed");
                                    play = [];
                                    thisTeamCallback();
                                }
                            });
                        });
                    }
                }, function afterAllTppUserOrgs() {
                    console.log(team1.name + " " + homeScore + " - " + awayScore + " " + team2.name);
                });
            });
        });
    });
}

function eventTeam() {
    return new Promise(function (resolve, reject) {
        var playTotal = getRandomNumber(1, 10);
        console.log("Total plays in the Match: " + playTotal);
        while (playTotal !== 0) {
            eventTeams.push(getRandomNumber(0, 1));
            playTotal--;
            if (playTotal === 0) {
                resolve();
            }
        }
    });
}

function goalScored(team1, team2, whichTeam) {
    return new Promise(function (resolve, reject) {
        var score = 0;
        async.eachSeries(play, function eachEvent(playEvent, playEventCallback) {
            console.log(playEvent);
            if (score === -1) {
                playEventCallback();
            } else {
                if (playEvent === "throwin" || playEvent === "corner" || playEvent === "goalkick" || playEvent === "freekick" || playEvent === "pass" || playEvent === "cross") {
                    if (whichTeam.players[getRandomNumber(1, 10)].skill.passing > getRandomNumber(0, 100)) {
                        playEventCallback();
                    } else {
                        score = -1;
                        resolve(score);
                    }
                } else if (playEvent === "penalty" || playEvent === "shoot") {
                    if (whichTeam.players[getRandomNumber(1, 10)].skill.shooting > getRandomNumber(0, 100)) {
                        console.log("goal scored");
                        score++;
                        resolve(score);
                    } else {
                        console.log("shot missed");
                        resolve(score);
                    }
                }
            }
        }, function afterAllEvents() {
            resolve(score);
        });
    });
}

function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}

function generatePlay() {
    return new Promise(function (resolve, reject) {
        var startEvent = eventStart[getRandomNumber(0, 4)];
        play.push(startEvent);
        if (startEvent === "penalty") {
            resolve(play);
        } else {
            newEventInPlay(function () {
                resolve(play);
            });
        }
    });
}

function newEventInPlay(callback) {
    newEvent = events[getRandomNumber(0, 2)];
    play.push(newEvent);
    if (newEvent === "shoot") {
        callback();
    } else {
        newEventInPlay(callback);
    }
}

function resetVars() {
    homeScore = 0;
    awayScore = 0;
}

function readFile(filePath) {
    return new Promise(function (resolve, reject) {
        fs.readFile(filePath, 'utf8', function (err, data) {
            if (err) {
                reject(err);
            } else {
                data = JSON.parse(data);
                resolve(data);
            }
        })
    });

Added tackling and saving to the formula

My final thought was how I could incorporate the defensive teams statistics to make it more of a two team game. To do this, when a player misses a pass or cross etc. We see if they were tackled or intercepted after the failure.
If yes, the particular play ends with no further actions. If not, the play continues. When a player shoots successfully (the rating is higher than the random number generated between 0 and 100) we see if the keeper was able to save it. To alleviate the bias this gives against scoring, we play many more events. In this example, 30.
var fs = require("fs");
var async = require("async");
var eventStart = ["throwin", "corner", "goalkick", "freekick", "penalty"];
var events = ["pass", "cross", "shoot"];
var homeScore = 0;
var awayScore = 0;
var play = [];
var eventTeams = [];

playMatch("teams/team1.json", "teams/team2.json");

function playMatch(team1Config, team2Config) {
        eventTeam().then(function () {
            readFile(team1Config).then(function (team1) {
                readFile(team2Config).then(function (team2) {
                    console.log("Todays Match is: " + team1.name + " vs " + team2.name);
                    async.eachSeries(eventTeams, function eachTeam(thisTeam, thisTeamCallback) {
                        //console.log("--------------------");
                        //console.log("Starting Play");
                        if (thisTeam === 0) {
                            generatePlay().then(function () {
                                goalScored(team1, team2).then(function (score) {
                                    if (score === 1) {
                                        //console.log(team1.name + " scored");
                                        homeScore++;
                                        play = [];
                                        thisTeamCallback();
                                    } else {
                                        //console.log(team1.name + " missed");
                                        play = [];
                                        thisTeamCallback();
                                    }
                                });
                            });
                        } else {
                            generatePlay().then(function () {
                                goalScored(team2, team1).then(function (score) {
                                    if (score === 1) {
                                        //console.log(team2.name + " scored");
                                        awayScore++;
                                        play = [];
                                        thisTeamCallback();
                                    } else {
                                        //console.log(team2.name + " missed");
                                        play = [];
                                        thisTeamCallback();
                                    }
                                });
                            });
                        }
                    }, function afterAllTppUserOrgs() {
                        console.log(team1.name + " " + homeScore + " - " + awayScore + " " + team2.name);
                        resetVars();
                        resolve();
                    });
                });
            });
        });
}

function eventTeam() {
    return new Promise(function (resolve, reject) {
        var playTotal = getRandomNumber(1, 30);
        //console.log("Total plays in the Match: " + playTotal);
        while (playTotal !== 0) {
            eventTeams.push(getRandomNumber(0, 1));
            playTotal--;
            if (playTotal === 0) {
                resolve();
            }
        }
    });
}

function goalScored(whichTeam, oppTeam) {
    return new Promise(function (resolve, reject) {
        var score = 0;
        async.eachSeries(play, function eachEvent(playEvent, playEventCallback) {
            var thisPlayer = whichTeam.players[getRandomNumber(1, 10)];
            var oppPlayer = oppTeam.players[getRandomNumber(0, 10)];
            //console.log(thisPlayer.name + " (" + thisPlayer.position + ")");
            //console.log(playEvent);
            if (score === -1) {
                playEventCallback();
            } else {
                if (playEvent === "throwin" || playEvent === "corner" || playEvent === "goalkick" || playEvent === "freekick" || playEvent === "pass" || playEvent === "cross") {
                    if (thisPlayer.skill.passing > getRandomNumber(0, 100)) {
                        playEventCallback();
                    } else {
                        //console.log("Retain play?");
                        //console.log("Challenge made by: " + oppPlayer.name + " (" + oppPlayer.position + ")");
                        if (oppPlayer.skill.tackling > getRandomNumber(0, 100)) {
                            //console.log("tackled. posession lost");
                            score = -1;
                            resolve(score);
                        } else {
                            //console.log("retained possession. Continue play");
                            playEventCallback();
                        }
                    }
                } else if (playEvent === "penalty" || playEvent === "shoot") {
                    if (thisPlayer.skill.shooting > getRandomNumber(0, 100)) {
                        if (oppTeam.players[0].skill.saving > getRandomNumber(0, 100)) {
                            //console.log("shot saved by " + oppTeam.players[0].name + " (" + oppTeam.players[0].position + ")");
                            resolve(score);
                        } else {
                            //console.log("goal conceeded by " + oppTeam.players[0].name + " (" + oppTeam.players[0].position + ")");
                            //console.log("goal scored by " + thisPlayer.name + " (" + thisPlayer.position + ")");
                            score++;
                            resolve(score);
                        }
                    } else {
                        //console.log("shot missed");
                        resolve(score);
                    }
                }
            }
        }, function afterAllEvents() {
            resolve(score);
        });
    });
}

function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}

function generatePlay() {
    return new Promise(function (resolve, reject) {
        var startEvent = eventStart[getRandomNumber(0, 4)];
        play.push(startEvent);
        if (startEvent === "penalty") {
            resolve(play);
        } else {
            newEventInPlay(function () {
                resolve(play);
            });
        }
    });
}

function newEventInPlay(callback) {
    newEvent = events[getRandomNumber(0, 2)];
    play.push(newEvent);
    if (newEvent === "shoot") {
        callback();
    } else {
        newEventInPlay(callback);
    }
}

function resetVars() {
    homeScore = 0;
    awayScore = 0;
    play = [];
    eventTeams = [];
}

function readFile(filePath) {
    return new Promise(function (resolve, reject) {
        fs.readFile(filePath, 'utf8', function (err, data) {
            if (err) {
                reject(err);
            } else {
                data = JSON.parse(data);
                resolve(data);
            }
        })
    });
}

What next?

The next step for this project would require much more intelligent movement, ratings, and event play for the user. I'm planning on developing this over the next few months and with any luck will have something to open source by the middle of next year.
However, this gives quite a good amount of data and events already. At the bottom I will show the output we see using this bit of code. However, if you have used, or adapted this code. Feel free to get in touch and let me know how it goes by email.
Good Luck!
Output:
Total plays in the Match: 22
Todays Match is: Brewers vs Arsenal
--------------------
Starting Play
Jim Boden (CB)
penalty
penalty on target by Jim Boden (CB)
goal conceeded by Bill Gallagher (GK)
Arsenal scored
--------------------
Starting Play
Gregory Gallagher (LM)
throwin
Retain play?
Challenge made by: Sid Boden (RB)
tackled. posession lost
Brewers missed
--------------------
Starting Play
Louise Boden (ST)
throwin
Retain play?
Challenge made by: Bill Gallagher (GK)
retained possession. Continue play
Arthur Boden (CM)
cross
Retain play?
Challenge made by: Cameron Gallagher (CM)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Gregory Boden (LM)
penalty
penalty on target by Gregory Boden (LM)
penalty saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Aiden Boden (ST)
penalty
penalty on target by Aiden Boden (ST)
goal conceeded by Bill Gallagher (GK)
Arsenal scored
--------------------
Starting Play
George Boden (CB)
goalkick
Retain play?
Challenge made by: Louise Peverley (ST)
retained possession. Continue play
Sid Boden (RB)
shoot
goal conceeded by Bill Gallagher (GK)
goal scored by Sid Boden (RB)
Arsenal scored
--------------------
Starting Play
Louise Boden (ST)
corner
Retain play?
Challenge made by: Aiden Gallagher (ST)
retained possession. Continue play
Tanisha Boden (RM)
shoot
shot saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Gregory Boden (LM)
freekick
Gregory Boden (LM)
cross
Cameron Boden (CM)
cross
Gregory Boden (LM)
cross
Sid Boden (RB)
pass
Arthur Boden (CM)
shoot
goal conceeded by Bill Gallagher (GK)
goal scored by Arthur Boden (CM)
Arsenal scored
--------------------
Starting Play
Jim Boden (CB)
goalkick
Retain play?
Challenge made by: Arthur Gallagher (CM)
retained possession. Continue play
George Boden (CB)
pass
Tanisha Boden (RM)
cross
Retain play?
Challenge made by: Cameron Gallagher (CM)
retained possession. Continue play
Aiden Boden (ST)
pass
Gregory Boden (LM)
pass
Fred Boden (LB)
pass
Retain play?
Challenge made by: Cameron Gallagher (CM)
retained possession. Continue play
Sid Boden (RB)
pass
Arthur Boden (CM)
cross
Retain play?
Challenge made by: Bill Gallagher (GK)
retained possession. Continue play
Jim Boden (CB)
cross
Retain play?
Challenge made by: Jim Gallagher (CB)
retained possession. Continue play
Gregory Boden (LM)
shoot
goal conceeded by Bill Gallagher (GK)
goal scored by Gregory Boden (LM)
Arsenal scored
--------------------
Starting Play
Cameron Gallagher (CM)
throwin
Gregory Gallagher (LM)
cross
Retain play?
Challenge made by: Tanisha Boden (RM)
tackled. posession lost
Brewers missed
--------------------
Starting Play
Louise Boden (ST)
goalkick
Cameron Boden (CM)
pass
George Boden (CB)
pass
Retain play?
Challenge made by: Louise Peverley (ST)
retained possession. Continue play
Fred Boden (LB)
shoot
shot missed
Arsenal missed
--------------------
Starting Play
Jim Boden (CB)
freekick
Jim Boden (CB)
cross
Retain play?
Challenge made by: Aiden Gallagher (ST)
retained possession. Continue play
Tanisha Boden (RM)
pass
Retain play?
Challenge made by: Tanisha Gallagher (RM)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Aiden Boden (ST)
corner
George Boden (CB)
cross
Louise Boden (ST)
cross
Retain play?
Challenge made by: George Gallagher (CB)
retained possession. Continue play
Arthur Boden (CM)
shoot
shot saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Sid Gallagher (RB)
corner
Sid Gallagher (RB)
shoot
goal conceeded by Bill Boden (GK)
goal scored by Sid Gallagher (RB)
Brewers scored
--------------------
Starting Play
George Gallagher (CB)
freekick
Gregory Gallagher (LM)
cross
Retain play?
Challenge made by: Louise Boden (ST)
tackled. posession lost
Brewers missed
--------------------
Starting Play
Sid Gallagher (RB)
penalty
penalty on target by Sid Gallagher (RB)
goal conceeded by Bill Boden (GK)
Brewers scored
--------------------
Starting Play
Tanisha Boden (RM)
corner
Sid Boden (RB)
cross
Arthur Boden (CM)
cross
Retain play?
Challenge made by: Jim Gallagher (CB)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Arthur Boden (CM)
goalkick
Retain play?
Challenge made by: Tanisha Gallagher (RM)
retained possession. Continue play
Aiden Boden (ST)
shoot
shot saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Arthur Boden (CM)
corner
Retain play?
Challenge made by: Louise Peverley (ST)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Tanisha Gallagher (RM)
corner
Sid Gallagher (RB)
shoot
shot saved by Bill Boden (GK)
Brewers missed
--------------------
Starting Play
Arthur Boden (CM)
corner
Tanisha Boden (RM)
pass
George Boden (CB)
shoot
shot missed
Arsenal missed
--------------------
Starting Play
Sid Boden (RB)
throwin
Tanisha Boden (RM)
shoot
shot missed
Arsenal missed
Brewers 2 - 5 Arsenal

Saturday, 7 October 2017

Deploying a Node JS application using IBM Bluemix UI

The big moment. After several weeks I had put together a dashboard for myself to include;

  • My Local Teams last two games, next game and two games after
  • Next trains from Waterloo to Winchester
  • Journey time from Hursley to Winchester
  • Last five of my home timeline tweets
  • Share prices
I also added, off the entry page;
  • Cinema time for two of my nearby cinemas
  • An old JS application I created that lets you find out how many cities you have visited.
It did what I wanted it to do, but I had to have my laptop running to view it.
My next idea was to be able to show it online whenever I wanted so I could access it as and when I liked. At first I considered using a web hosting service like 1&1 (who I have another web address with) but decided I would instead make use of the IBM Bluemix service.

IBM Bluemix has a node SDK which has a plan of £0.0424 GBP/GB-Hour.

There are a few developerWorks articles that describe deploying a node js application:

An alternative method to the command line as shown in the three examples above, is to use the bluemix User Interface. Which is what I opted for this time, having created several application using the CLI before.

 1. Create a "Cloud Foundry Apps" -> SDK for Node.js as seen in the image above.
 2. Give your application a name, select the hosting country and select "create"

 3. The application will build and then start. You need to wait for it to show as "running" at which point it will be available. In my example, this is true for the url: https://mytest012345.mybluemix.net
 4. Within the applications bluemix "Application Details" page we can go to "Overview" and "enable" continuous delivery.
 5. This creates a toolchain which include a git repo and an eclipse browser.
 6. Select the eclipse tool and we can browse the files in the editor
standard SDK for node JS project
 7. Delete the public folder
 8. Zip your own public folder as well as your dependencies such as your node modules.
 9. Right click on your project folder and select "import -> File or Zip archive" and select the zip file you just created.
 10. The zip will be unzipped and the files and directories unpackaged into the project.
 11. Replace the "package.json" with your own and replace the README.md.
 12. Update the app.js file with your path routes, variables and dependencies
//ADD YOUR VARIABLES
//var abc = require("abc");
/*eslint-env node*/
//------------------------------------------------------------------------------
// node.js starter application for Bluemix
//------------------------------------------------------------------------------
// This application uses express as its web server
// for more info, see: http://expressjs.com
var express = require('express');
// cfenv provides access to your Cloud Foundry environment
// for more info, see: https://www.npmjs.com/package/cfenv
var cfenv = require('cfenv');
// create a new express server
var app = express();
// serve the files out of ./public as our main files
app.use(express.static(__dirname + '/public'));
/*app.get(/route1, function(req, res){

}); REPEAT WITH ALL ROUTES*/
// get the app environment from Cloud Foundry
var appEnv = cfenv.getAppEnv();
// start server on the specified port and binding host
app.listen(appEnv.port, '0.0.0.0', function() {
  // print a message when the server starts listening
  console.log("server starting on " + appEnv.url);
});
 13. We now need to commit the changes we have made. On the left panel, select the "git" option and commit the changes we have just made.
The second button is a link to commit changes to our git repository
 14. We can now redeploy the application from the eclipse toolkit using the play button.
The green circle shows the application is running, if red it would indicate an error. Next to this is a play button which deploys the app, the stop button stops the application, the third button opens the application in a new tab, second from right opens the logs and the final button opens the bluemix "application overview" page.

The application should now be running again.
*I would recommend for ease and speed to follow the command line functionality described in the links presented earlier*

So... I now have my application available.

Live Dashboard
For an added extra I opened the application on my phone:
In iOS we can press the box with an arrow coming out of it and we can save the page to our home screen.
This makes it available on the home screen making for quick access.

So, the dashboard is up, I can access it and it's being hosted on Bluemix. All is good with the world and another project is complete.

Have some questions? Think you can suggest some improvements? Feel free to email me here.
Alternatively leave a comment below. 

Quote API with node JS

As Winston Churchill once said: "It is a good thing for an uneducated man to read books of quotations."

Having almost completed all the content I want for my dashboard I decided to add a little more to the title which currently just says "Gally Home Dashboard". I remember back at my old Secondary School - de Ferrers Specialist Technology College we used to have an intranet that gave a "thought of the day." So I started looking at how I can add quotes to my dashboard.
Boring Title Page
Option 1: Create a JSON file that contains quotations I like.
var quotes= [{
 "name": "Winston Churchill",
 "quote": "It is a good thing for an uneducated man to read books of quotations."
},
...];
We can then access these quotes by accessing a random part of the array.
getQuote(randNo){
  var thisQuote = JSON.parse(quotes[randNo]);
  var returnString = thisQuote.name + ": " + thisQuote.quote;
  return returnString;
}
There are two downsides to this. 1. I'd have to populate a huge JSON 2. It would be full of quotes I already know and would leave me in a "quote bubble". Not very educational to the uneducated man.

Option 2: Don't reinvent the wheel. Use an API.
There are several APIs we can use;
 * Quotes on Design - This allows us to decide a filter on the order and the number of results.
JS:
$.getJSON("https://quotesondesign.com/wp-json/posts?filter[orderby]=rand&filter[posts_per_page]=1&callback=", function(a) {
  $("body").append(a[0].content + "<p>&mdash; " + a[0].title + "</p>")
});
Example from the webpage
Curl:
curl -X GET "https://quotesondesign.com/wp-json/posts?filter=rand&filter=1"
* Forismatic - Similar to quotes on design but also includes "expressions". Neither appears to have rate limits or need an authentication code to be used!
Results can come in
Curl:
curl -X GET "https://api.forismatic.com/api/1.0/?method=getQuote&key=&format=json&lang=en
The language can be set to english or russian (?)
key can be set to blank which gives a random quote.
response format comes in: json, jsonp, xml, html and plain text.

Output for english, random key and different formats:
html: 
<blockquote><strong><cite>We are what we repeatedly do. Excellence, then, is not an act, but a habit. </cite></strong><br/><small>Aristotle <address></address></small><blockquote>
json:
{"quoteText":"We know the truth, not only by the reason, but by the heart. ", "quoteAuthor":"Blaise Pascal", "senderName":"", "senderLink":"", "quoteLink":"http://forismatic.com/en/b917914ba7/"}
text:
You do not become good by trying to be good, but by finding the goodness that is already within you. (Eckhart Tolle) 
The second comes with a bit more freedom. In the end I made it match how I was doing requests throughout the rest of the dashboard (please see my other posts) and then self formatting to how I wanted it to look on the client side.

My Basic Flow:
 - Use request npm to match the curl command above (using json)
 - Send the json response to the client
 - Parse the json and format the output in the client JS
document.getElementById("randomQuote").innerHTML = quote.quoteAuthor + ": " + quote.quoteText; 
 - Some responses had no author, so we attribute them to "anon"
 if (quote.quoteAuthor === "") {
   quote.quoteAuthor = "Anon";
}

To what I believe are great results!


Any thoughts? Feel free to comment below. 

Sunday, 1 October 2017

Cinema Time API

Continuing on from the earlier success of my dashboard API's to date: Twitter, National Rail Enquiries as well as a google finance integration and a home made Burton Albion score API. Now, on request from Louise I have looked to display all the films and film times for the local cinema.

I did a quick google of "Cinema API" and came up with something called "moviesapi" which you can find here: http://moviesapi.herokuapp.com which is a screen scraping API for https://www.findanyfilm.com which is a website that lets film fans find how to download, watch, buy and rent films - all above the board may I add.

The API itself is a non authenticated, JSON returning API that appears to have no rate limit (or so far as I can tell). It's quite a simple idea, there are two GET API's, find a nearby cinema and list films from a specific cinema using the id we can get from the nearby cinema API. 

In curl we can do "curl -X GET "http://moviesapi.herokuapp.com/cinemas/find/SO23"" (my nearby area)
which returns the following data (I've cut it down for readability):
[{"venue_id":"7144","name":"VUE Eastleigh, Eastleigh","address":"VUE Eastleigh, Swan Centre, Wells Place, Eastleigh, Eastleigh, SO50 5SF","url":"https://www.myvue.com/cinema/eastleigh?SC_CMP=AFF_indtrust","distance":""}]
We can then use this "venue_id" to get the selected cinemas film list for all times on the same day. The same day part is a significant restriction, but works for a dashboard that only wants to show this reduced information.

In curl we can do "curl -X GET "http://moviesapi.herokuapp.com/cinemas/7144/showings"" which returns the following film data (I've cut it down again for readability):
[{"title":"The Emoji Movie","link":"","time":["10:00","12:15","13:20","15:10"]},{"title":"The Nut Job 2: Nutty By Nature","link":"","time":["10:00","11:45"]}]
For the purpose of displaying the full information for the selected cinema I have created a server side http request using the request node module (npm). My code looks like a getFilmTimes function which is itself a promise, I create the request options and which has a url and a method, send the the request and then resolve the parsed JSON body.
function getFilmTimes(venueId) {
return new Promise(function (resolve, reject) {
var postOptions = {
url: 'http://moviesapi.herokuapp.com/cinemas/' + venueId + '/showings',
method: 'GET'
};
request(postOptions, function cb(err, res, body) {
if (err) {
log.dashErr(JSON.stringify(err))
reject();
}
if (res.statusCode === 200) {
resolve(JSON.parse(body));
}
});
});
}
I want to get all the information as a single string (html formatted) so that I can push it to the browser and display on new lines each film and all the times associated to it.

To do this, I want to cycle through the results and add them to a string variable using the "async" node module (npm), it will also mean cycling the film times to add them to the end of the derived strings. See the code below:
var films = JSON.parse(body);
var sendFilms = "";
async.eachSeries(films, function cycleThroughFilms(currentFilm, filmCallback) {
var filmTimes = currentFilm.time;
sendFilms = sendFilms + currentFilm.title + ": ";
async.eachSeries(filmTimes, function cycleThroughTimes(currentTime, currentTimeCallback) {
sendFilms = sendFilms + currentTime + " ";
currentTimeCallback();
}, function afterTimesDone() {
sendFilms = sendFilms + "<br>";
});
filmCallback();
}, function afterFilmsProcessed(err) {
resolve(sendFilms);
});
The above code takes an array "films" and cycles through each set of results. We add first the film title to our global "sendFilms" variable before cycling the times array within the film array. This cycles "filmTimes" which is the current films times array and adds the current time to the global variable. When we have cycled through the film times we add a "<br>" html break line tag and then continue to the next film using callbacks available.

Upon completion of all the films within the array I resolve the promise and send the string on - eventually to the browser.

This worked well but was displaying film times that had already passed so to fix this we can add a little function to get the current time and compare the times as we process them to see if they have already started.
...
async.eachSeries(filmTimes, function cycleThroughTimes(currentTime, currentTimeCallback) {
var timenow = new Date().toISOString().substr(11, 5);
if (currentTime > timenow) {
sendFilms = sendFilms + currentTime + " ";
}
currentTimeCallback();
}, function afterTimesDone() {
...
With that we're all done:
The last thought is that we could remove the films that have no more showings for the day. But I personally like to see what might be available the next day throughout the day. I'll see how it feels.

Any questions? Anything you would change/improve upon. Give me an email on aiden.g@live.co.uk or leave a comment. 


Tuesday, 26 September 2017

Using twitters user timeline API in Node JS

Having continued to create my dashboard that allows me to access information thats useful to me in a single, easy to access location, I began to think about how I could get my twitter timeline to display on screen (messages that I post and messages people I follow post).

Normally I like to try to just do a curl GET of the webpage I'm looking to use but twitter requires an oauth key to do even a get on your own profile. *sigh*.

Since I've started developing Twitter have changed their entire web based content to a flash new website, a few new APIs and a new "feel".
New Twitter Site

The first thing we need to become a twitter developer is to create an application on https://apps.twitter.com which requires a login and for your account to have a linked mobile number. You'll need to give the application a name and a website address where the application will later ve accessible (this doesn't have to be real and can be changed later).

You will then be given application settings information complete with access token, and security keys, specifically within the "Keys and access tokens" tab.
Application Settings
-----------------------------------------------------
What I specifically wanted to do in my dashboard was display a few of the latest tweets from my home timeline which was surprisingly difficult to figure out as most use cases seem to be around mass production of tweets and making "bots" and a lot of the documentation for node module implementation on node js didn't really describe what was happening.

It's worth noting that when thinking about how I wanted to achieve this I had to consider twitters rate limiting for per access token usage. Which comes to 15 calls per application/access token per 15 minute window, meaning realistically we could make one call a minute.

Initially I just wanted to get set up and the easiest way to do so seemed to be with an existing node module called (wait for it...) twitter npm. https://www.npmjs.com/package/twitter. Following their instructions I took a plain old Node Js application and added the following code using the oauth tokens we get in our application settings:
var Twitter = require('twitter');
var client = new Twitter({
      consumer_key: 'XXX',
      consumer_secret: 'XXX',
      access_token_key: 'XXX',
      access_token_secret: 'XXX'
    });
At this point there are two options for receiving tweets from our timeline and this is to use the Stream API or GET API.

STREAM: Creates a single connection that receives data as it is pushed from twitter, this allows us to process the data in real time and not make continuous calls to twitter.

GET: Consists of standard HTTPS GET calls as and when we want to get information from twitter.

Whilst streaming gives us the most up to date data from twitter, it comes with additional complexity of having an open connection between your server and twitter that must be secured and kept alive which will generally require a bit more thought than an evenings coding. GET on the other hand suits my purpose well and allows me to call and handle data as and when I want.

For fullness I will show how to implement both:
----------------------------------------------------------------------------------------------------------
STREAM User Timeline:
client.stream('user', {track: 'GallagherAiden'}, function(stream) {
      stream.on('data', function(tweet) {
        console.log("got the following twitter message: " + JSON.stringify(tweet));
        var createdTime = tweet.created_at;
        createdTime = createdTime.substring(0,16);
        var tweetStream = createdTime + " " + tweet.user.name + " : " + tweet.text;
      });
      stream.on('error', function(error) {
        if(error){
          log.dashErr(error);
        }
         console.err("Could not get any twitter feeds");
        });
      });
    });
Here we use the twitter npm ".stream" functionality with the client defining that we want a "user"s stream and that that users handle is "GallagherAiden".

We then use "stream.on" to handle the incoming data which I then stringify and make suitable for my needs. i.e. in the following format:
Tue Sep 26 19:04 Gary Lineker - Gorgeous finish from @GarethBale11 gives Real Madrid the lead in Dortmund.
----------------------------------------------------------------------------------------------------------
GET User Timeline:
client.get('statuses/home_timeline', function(error, tweets, response) {
      if(error){
       console.log(error);
      }
var thisTweet = tweets[0];
var createdTime = thisTweet.created_at;
createdTime = createdTime.substring(0,16);
var tweetGet = createdTime + " " + thisTweet.user.name + " - " + thisTweet.text;
console.log(tweetGet);

Here we use 'statuses/home_timeline' which responds either with the tweets, an error or the response. This relates directly to the home timeline of the user who is linked to the application. There is no way to alter this.
The above code would give the same output as the stream in terms of formatting.
----------------------------------------------------------------------------------------------------------

The output of a single tweet is extremely large! 
{
    "created_at": "Mon Sep 25 06:22:53 +0000 2017",
    "id": 912200715552124900,
    "id_str": "912200715552124928",
    "text": "Photographing the Peak District in autumn \u2013 in pictures https://t.co/mHSVxMpNt2",
    "truncated": false,
    "entities": {
      "hashtags": [],
      "symbols": [],
      "user_mentions": [],
      "urls": [
        {
          "url": "https://t.co/mHSVxMpNt2",
          "expanded_url": "https://trib.al/hCIqP6P",
          "display_url": "trib.al/hCIqP6P",
          "indices": [
            56,
            79
          ]
        }
      ]
    },
    "source": "<a href=\"http://www.socialflow.com\" rel=\"nofollow\">SocialFlow</a>",
    "in_reply_to_status_id": null,
    "in_reply_to_status_id_str": null,
    "in_reply_to_user_id": null,
    "in_reply_to_user_id_str": null,
    "in_reply_to_screen_name": null,
    "user": {
      "id": 87818409,
      "id_str": "87818409",
      "name": "The Guardian",
      "screen_name": "guardian",
      "location": "London",
      "description": "The need for independent journalism has never been greater. Become a Guardian supporter: https://t.co/EWg9aqA7PQ",
      "url": "https://t.co/c53pnmnuIT",
      "entities": {
        "url": {
          "urls": [
            {
              "url": "https://t.co/c53pnmnuIT",
              "expanded_url": "https://www.theguardian.com",
              "display_url": "theguardian.com",
              "indices": [
                0,
                23
              ]
            }
          ]
        },
        "description": {
          "urls": [
            {
              "url": "https://t.co/EWg9aqA7PQ",
              "expanded_url": "http://gu.com/supporter/twitter",
              "display_url": "gu.com/supporter/twit\u2026",
              "indices": [
                89,
                112
              ]
            }
          ]
        }
      },
      "protected": false,
      "followers_count": 6791634,
      "friends_count": 1113,
      "listed_count": 54290,
      "created_at": "Thu Nov 05 23:49:19 +0000 2009",
      "favourites_count": 152,
      "utc_offset": 3600,
      "time_zone": "London",
      "geo_enabled": false,
      "verified": true,
      "statuses_count": 378671,
      "lang": "en",
      "contributors_enabled": false,
      "is_translator": false,
      "is_translation_enabled": true,
      "profile_background_color": "FFFFFF",
      "profile_background_image_url": "http://pbs.twimg.com/profile_background_images/704160749/ff996aa3bc2009a2f9b97cdd43e8b5b7.png",
      "profile_background_image_url_https": "https://pbs.twimg.com/profile_background_images/704160749/ff996aa3bc2009a2f9b97cdd43e8b5b7.png",
      "profile_background_tile": false,
      "profile_image_url": "http://pbs.twimg.com/profile_images/877153924637175809/deHwf3Qu_normal.jpg",
      "profile_image_url_https": "https://pbs.twimg.com/profile_images/877153924637175809/deHwf3Qu_normal.jpg",
      "profile_banner_url": "https://pbs.twimg.com/profile_banners/87818409/1491899430",
      "profile_link_color": "005789",
      "profile_sidebar_border_color": "FFFFFF",
      "profile_sidebar_fill_color": "CAE3F3",
      "profile_text_color": "333333",
      "profile_use_background_image": false,
      "has_extended_profile": false,
      "default_profile": false,
      "default_profile_image": false,
      "following": true,
      "follow_request_sent": false,
      "notifications": false,
      "translator_type": "regular"
    },
    "geo": null,
    "coordinates": null,
    "place": null,
    "contributors": null,
    "is_quote_status": false,
    "retweet_count": 1,
    "favorite_count": 3,
    "favorited": false,
    "retweeted": false,
    "possibly_sensitive": false,
    "possibly_sensitive_appealable": false,
    "lang": "en"
  }
 ----------------------------------------------------------------------------------------------------------
I have implemented this in a way that calls from the server using a "setInterval" ever 60 seconds which keeps me within my rate limit:
setInterval(function(){
  getTwitterFeed();
}, 60000);
I then do a web client GET of this data in order to populate my dashboard. I'm currently working with the last five tweets and will probably post how I'm doing that in a later post.

For now. Adieu. I hope you enjoyed reading, feel free to tell me where I'm wrong and how I can improve. you can email me at aiden.g@live.co.uk  or leave me a comment.

Twitter Feed on my dashboard, reloading every sixty seconds