Saturday, 7 October 2017

Deploying a Node JS application using IBM Bluemix UI

The big moment. After several weeks I had put together a dashboard for myself to include;

  • My Local Teams last two games, next game and two games after
  • Next trains from Waterloo to Winchester
  • Journey time from Hursley to Winchester
  • Last five of my home timeline tweets
  • Share prices
I also added, off the entry page;
  • Cinema time for two of my nearby cinemas
  • An old JS application I created that lets you find out how many cities you have visited.
It did what I wanted it to do, but I had to have my laptop running to view it.
My next idea was to be able to show it online whenever I wanted so I could access it as and when I liked. At first I considered using a web hosting service like 1&1 (who I have another web address with) but decided I would instead make use of the IBM Bluemix service.

IBM Bluemix has a node SDK which has a plan of £0.0424 GBP/GB-Hour.

There are a few developerWorks articles that describe deploying a node js application:

An alternative method to the command line as shown in the three examples above, is to use the bluemix User Interface. Which is what I opted for this time, having created several application using the CLI before.

 1. Create a "Cloud Foundry Apps" -> SDK for Node.js as seen in the image above.
 2. Give your application a name, select the hosting country and select "create"

 3. The application will build and then start. You need to wait for it to show as "running" at which point it will be available. In my example, this is true for the url: https://mytest012345.mybluemix.net
 4. Within the applications bluemix "Application Details" page we can go to "Overview" and "enable" continuous delivery.
 5. This creates a toolchain which include a git repo and an eclipse browser.
 6. Select the eclipse tool and we can browse the files in the editor
standard SDK for node JS project
 7. Delete the public folder
 8. Zip your own public folder as well as your dependencies such as your node modules.
 9. Right click on your project folder and select "import -> File or Zip archive" and select the zip file you just created.
 10. The zip will be unzipped and the files and directories unpackaged into the project.
 11. Replace the "package.json" with your own and replace the README.md.
 12. Update the app.js file with your path routes, variables and dependencies
//ADD YOUR VARIABLES
//var abc = require("abc");
/*eslint-env node*/
//------------------------------------------------------------------------------
// node.js starter application for Bluemix
//------------------------------------------------------------------------------
// This application uses express as its web server
// for more info, see: http://expressjs.com
var express = require('express');
// cfenv provides access to your Cloud Foundry environment
// for more info, see: https://www.npmjs.com/package/cfenv
var cfenv = require('cfenv');
// create a new express server
var app = express();
// serve the files out of ./public as our main files
app.use(express.static(__dirname + '/public'));
/*app.get(/route1, function(req, res){

}); REPEAT WITH ALL ROUTES*/
// get the app environment from Cloud Foundry
var appEnv = cfenv.getAppEnv();
// start server on the specified port and binding host
app.listen(appEnv.port, '0.0.0.0', function() {
  // print a message when the server starts listening
  console.log("server starting on " + appEnv.url);
});
 13. We now need to commit the changes we have made. On the left panel, select the "git" option and commit the changes we have just made.
The second button is a link to commit changes to our git repository
 14. We can now redeploy the application from the eclipse toolkit using the play button.
The green circle shows the application is running, if red it would indicate an error. Next to this is a play button which deploys the app, the stop button stops the application, the third button opens the application in a new tab, second from right opens the logs and the final button opens the bluemix "application overview" page.

The application should now be running again.
*I would recommend for ease and speed to follow the command line functionality described in the links presented earlier*

So... I now have my application available.

Live Dashboard
For an added extra I opened the application on my phone:
In iOS we can press the box with an arrow coming out of it and we can save the page to our home screen.
This makes it available on the home screen making for quick access.

So, the dashboard is up, I can access it and it's being hosted on Bluemix. All is good with the world and another project is complete.

Have some questions? Think you can suggest some improvements? Feel free to email me here.
Alternatively leave a comment below. 

Quote API with node JS

As Winston Churchill once said: "It is a good thing for an uneducated man to read books of quotations."

Having almost completed all the content I want for my dashboard I decided to add a little more to the title which currently just says "Gally Home Dashboard". I remember back at my old Secondary School - de Ferrers Specialist Technology College we used to have an intranet that gave a "thought of the day." So I started looking at how I can add quotes to my dashboard.
Boring Title Page
Option 1: Create a JSON file that contains quotations I like.
var quotes= [{
 "name": "Winston Churchill",
 "quote": "It is a good thing for an uneducated man to read books of quotations."
},
...];
We can then access these quotes by accessing a random part of the array.
getQuote(randNo){
  var thisQuote = JSON.parse(quotes[randNo]);
  var returnString = thisQuote.name + ": " + thisQuote.quote;
  return returnString;
}
There are two downsides to this. 1. I'd have to populate a huge JSON 2. It would be full of quotes I already know and would leave me in a "quote bubble". Not very educational to the uneducated man.

Option 2: Don't reinvent the wheel. Use an API.
There are several APIs we can use;
 * Quotes on Design - This allows us to decide a filter on the order and the number of results.
JS:
$.getJSON("https://quotesondesign.com/wp-json/posts?filter[orderby]=rand&filter[posts_per_page]=1&callback=", function(a) {
  $("body").append(a[0].content + "<p>&mdash; " + a[0].title + "</p>")
});
Example from the webpage
Curl:
curl -X GET "https://quotesondesign.com/wp-json/posts?filter=rand&filter=1"
* Forismatic - Similar to quotes on design but also includes "expressions". Neither appears to have rate limits or need an authentication code to be used!
Results can come in
Curl:
curl -X GET "https://api.forismatic.com/api/1.0/?method=getQuote&key=&format=json&lang=en
The language can be set to english or russian (?)
key can be set to blank which gives a random quote.
response format comes in: json, jsonp, xml, html and plain text.

Output for english, random key and different formats:
html: 
<blockquote><strong><cite>We are what we repeatedly do. Excellence, then, is not an act, but a habit. </cite></strong><br/><small>Aristotle <address></address></small><blockquote>
json:
{"quoteText":"We know the truth, not only by the reason, but by the heart. ", "quoteAuthor":"Blaise Pascal", "senderName":"", "senderLink":"", "quoteLink":"http://forismatic.com/en/b917914ba7/"}
text:
You do not become good by trying to be good, but by finding the goodness that is already within you. (Eckhart Tolle) 
The second comes with a bit more freedom. In the end I made it match how I was doing requests throughout the rest of the dashboard (please see my other posts) and then self formatting to how I wanted it to look on the client side.

My Basic Flow:
 - Use request npm to match the curl command above (using json)
 - Send the json response to the client
 - Parse the json and format the output in the client JS
document.getElementById("randomQuote").innerHTML = quote.quoteAuthor + ": " + quote.quoteText; 
 - Some responses had no author, so we attribute them to "anon"
 if (quote.quoteAuthor === "") {
   quote.quoteAuthor = "Anon";
}

To what I believe are great results!


Any thoughts? Feel free to comment below. 

Sunday, 1 October 2017

Cinema Time API

Continuing on from the earlier success of my dashboard API's to date: Twitter, National Rail Enquiries as well as a google finance integration and a home made Burton Albion score API. Now, on request from Louise I have looked to display all the films and film times for the local cinema.

I did a quick google of "Cinema API" and came up with something called "moviesapi" which you can find here: http://moviesapi.herokuapp.com which is a screen scraping API for https://www.findanyfilm.com which is a website that lets film fans find how to download, watch, buy and rent films - all above the board may I add.

The API itself is a non authenticated, JSON returning API that appears to have no rate limit (or so far as I can tell). It's quite a simple idea, there are two GET API's, find a nearby cinema and list films from a specific cinema using the id we can get from the nearby cinema API. 

In curl we can do "curl -X GET "http://moviesapi.herokuapp.com/cinemas/find/SO23"" (my nearby area)
which returns the following data (I've cut it down for readability):
[{"venue_id":"7144","name":"VUE Eastleigh, Eastleigh","address":"VUE Eastleigh, Swan Centre, Wells Place, Eastleigh, Eastleigh, SO50 5SF","url":"https://www.myvue.com/cinema/eastleigh?SC_CMP=AFF_indtrust","distance":""}]
We can then use this "venue_id" to get the selected cinemas film list for all times on the same day. The same day part is a significant restriction, but works for a dashboard that only wants to show this reduced information.

In curl we can do "curl -X GET "http://moviesapi.herokuapp.com/cinemas/7144/showings"" which returns the following film data (I've cut it down again for readability):
[{"title":"The Emoji Movie","link":"","time":["10:00","12:15","13:20","15:10"]},{"title":"The Nut Job 2: Nutty By Nature","link":"","time":["10:00","11:45"]}]
For the purpose of displaying the full information for the selected cinema I have created a server side http request using the request node module (npm). My code looks like a getFilmTimes function which is itself a promise, I create the request options and which has a url and a method, send the the request and then resolve the parsed JSON body.
function getFilmTimes(venueId) {
return new Promise(function (resolve, reject) {
var postOptions = {
url: 'http://moviesapi.herokuapp.com/cinemas/' + venueId + '/showings',
method: 'GET'
};
request(postOptions, function cb(err, res, body) {
if (err) {
log.dashErr(JSON.stringify(err))
reject();
}
if (res.statusCode === 200) {
resolve(JSON.parse(body));
}
});
});
}
I want to get all the information as a single string (html formatted) so that I can push it to the browser and display on new lines each film and all the times associated to it.

To do this, I want to cycle through the results and add them to a string variable using the "async" node module (npm), it will also mean cycling the film times to add them to the end of the derived strings. See the code below:
var films = JSON.parse(body);
var sendFilms = "";
async.eachSeries(films, function cycleThroughFilms(currentFilm, filmCallback) {
var filmTimes = currentFilm.time;
sendFilms = sendFilms + currentFilm.title + ": ";
async.eachSeries(filmTimes, function cycleThroughTimes(currentTime, currentTimeCallback) {
sendFilms = sendFilms + currentTime + " ";
currentTimeCallback();
}, function afterTimesDone() {
sendFilms = sendFilms + "<br>";
});
filmCallback();
}, function afterFilmsProcessed(err) {
resolve(sendFilms);
});
The above code takes an array "films" and cycles through each set of results. We add first the film title to our global "sendFilms" variable before cycling the times array within the film array. This cycles "filmTimes" which is the current films times array and adds the current time to the global variable. When we have cycled through the film times we add a "<br>" html break line tag and then continue to the next film using callbacks available.

Upon completion of all the films within the array I resolve the promise and send the string on - eventually to the browser.

This worked well but was displaying film times that had already passed so to fix this we can add a little function to get the current time and compare the times as we process them to see if they have already started.
...
async.eachSeries(filmTimes, function cycleThroughTimes(currentTime, currentTimeCallback) {
var timenow = new Date().toISOString().substr(11, 5);
if (currentTime > timenow) {
sendFilms = sendFilms + currentTime + " ";
}
currentTimeCallback();
}, function afterTimesDone() {
...
With that we're all done:
The last thought is that we could remove the films that have no more showings for the day. But I personally like to see what might be available the next day throughout the day. I'll see how it feels.

Any questions? Anything you would change/improve upon. Give me an email on aiden.g@live.co.uk or leave a comment. 


Tuesday, 26 September 2017

Using twitters user timeline API in Node JS

Having continued to create my dashboard that allows me to access information thats useful to me in a single, easy to access location, I began to think about how I could get my twitter timeline to display on screen (messages that I post and messages people I follow post).

Normally I like to try to just do a curl GET of the webpage I'm looking to use but twitter requires an oauth key to do even a get on your own profile. *sigh*.

Since I've started developing Twitter have changed their entire web based content to a flash new website, a few new APIs and a new "feel".
New Twitter Site

The first thing we need to become a twitter developer is to create an application on https://apps.twitter.com which requires a login and for your account to have a linked mobile number. You'll need to give the application a name and a website address where the application will later ve accessible (this doesn't have to be real and can be changed later).

You will then be given application settings information complete with access token, and security keys, specifically within the "Keys and access tokens" tab.
Application Settings
-----------------------------------------------------
What I specifically wanted to do in my dashboard was display a few of the latest tweets from my home timeline which was surprisingly difficult to figure out as most use cases seem to be around mass production of tweets and making "bots" and a lot of the documentation for node module implementation on node js didn't really describe what was happening.

It's worth noting that when thinking about how I wanted to achieve this I had to consider twitters rate limiting for per access token usage. Which comes to 15 calls per application/access token per 15 minute window, meaning realistically we could make one call a minute.

Initially I just wanted to get set up and the easiest way to do so seemed to be with an existing node module called (wait for it...) twitter npm. https://www.npmjs.com/package/twitter. Following their instructions I took a plain old Node Js application and added the following code using the oauth tokens we get in our application settings:
var Twitter = require('twitter');
var client = new Twitter({
      consumer_key: 'XXX',
      consumer_secret: 'XXX',
      access_token_key: 'XXX',
      access_token_secret: 'XXX'
    });
At this point there are two options for receiving tweets from our timeline and this is to use the Stream API or GET API.

STREAM: Creates a single connection that receives data as it is pushed from twitter, this allows us to process the data in real time and not make continuous calls to twitter.

GET: Consists of standard HTTPS GET calls as and when we want to get information from twitter.

Whilst streaming gives us the most up to date data from twitter, it comes with additional complexity of having an open connection between your server and twitter that must be secured and kept alive which will generally require a bit more thought than an evenings coding. GET on the other hand suits my purpose well and allows me to call and handle data as and when I want.

For fullness I will show how to implement both:
----------------------------------------------------------------------------------------------------------
STREAM User Timeline:
client.stream('user', {track: 'GallagherAiden'}, function(stream) {
      stream.on('data', function(tweet) {
        console.log("got the following twitter message: " + JSON.stringify(tweet));
        var createdTime = tweet.created_at;
        createdTime = createdTime.substring(0,16);
        var tweetStream = createdTime + " " + tweet.user.name + " : " + tweet.text;
      });
      stream.on('error', function(error) {
        if(error){
          log.dashErr(error);
        }
         console.err("Could not get any twitter feeds");
        });
      });
    });
Here we use the twitter npm ".stream" functionality with the client defining that we want a "user"s stream and that that users handle is "GallagherAiden".

We then use "stream.on" to handle the incoming data which I then stringify and make suitable for my needs. i.e. in the following format:
Tue Sep 26 19:04 Gary Lineker - Gorgeous finish from @GarethBale11 gives Real Madrid the lead in Dortmund.
----------------------------------------------------------------------------------------------------------
GET User Timeline:
client.get('statuses/home_timeline', function(error, tweets, response) {
      if(error){
       console.log(error);
      }
var thisTweet = tweets[0];
var createdTime = thisTweet.created_at;
createdTime = createdTime.substring(0,16);
var tweetGet = createdTime + " " + thisTweet.user.name + " - " + thisTweet.text;
console.log(tweetGet);

Here we use 'statuses/home_timeline' which responds either with the tweets, an error or the response. This relates directly to the home timeline of the user who is linked to the application. There is no way to alter this.
The above code would give the same output as the stream in terms of formatting.
----------------------------------------------------------------------------------------------------------

The output of a single tweet is extremely large! 
{
    "created_at": "Mon Sep 25 06:22:53 +0000 2017",
    "id": 912200715552124900,
    "id_str": "912200715552124928",
    "text": "Photographing the Peak District in autumn \u2013 in pictures https://t.co/mHSVxMpNt2",
    "truncated": false,
    "entities": {
      "hashtags": [],
      "symbols": [],
      "user_mentions": [],
      "urls": [
        {
          "url": "https://t.co/mHSVxMpNt2",
          "expanded_url": "https://trib.al/hCIqP6P",
          "display_url": "trib.al/hCIqP6P",
          "indices": [
            56,
            79
          ]
        }
      ]
    },
    "source": "<a href=\"http://www.socialflow.com\" rel=\"nofollow\">SocialFlow</a>",
    "in_reply_to_status_id": null,
    "in_reply_to_status_id_str": null,
    "in_reply_to_user_id": null,
    "in_reply_to_user_id_str": null,
    "in_reply_to_screen_name": null,
    "user": {
      "id": 87818409,
      "id_str": "87818409",
      "name": "The Guardian",
      "screen_name": "guardian",
      "location": "London",
      "description": "The need for independent journalism has never been greater. Become a Guardian supporter: https://t.co/EWg9aqA7PQ",
      "url": "https://t.co/c53pnmnuIT",
      "entities": {
        "url": {
          "urls": [
            {
              "url": "https://t.co/c53pnmnuIT",
              "expanded_url": "https://www.theguardian.com",
              "display_url": "theguardian.com",
              "indices": [
                0,
                23
              ]
            }
          ]
        },
        "description": {
          "urls": [
            {
              "url": "https://t.co/EWg9aqA7PQ",
              "expanded_url": "http://gu.com/supporter/twitter",
              "display_url": "gu.com/supporter/twit\u2026",
              "indices": [
                89,
                112
              ]
            }
          ]
        }
      },
      "protected": false,
      "followers_count": 6791634,
      "friends_count": 1113,
      "listed_count": 54290,
      "created_at": "Thu Nov 05 23:49:19 +0000 2009",
      "favourites_count": 152,
      "utc_offset": 3600,
      "time_zone": "London",
      "geo_enabled": false,
      "verified": true,
      "statuses_count": 378671,
      "lang": "en",
      "contributors_enabled": false,
      "is_translator": false,
      "is_translation_enabled": true,
      "profile_background_color": "FFFFFF",
      "profile_background_image_url": "http://pbs.twimg.com/profile_background_images/704160749/ff996aa3bc2009a2f9b97cdd43e8b5b7.png",
      "profile_background_image_url_https": "https://pbs.twimg.com/profile_background_images/704160749/ff996aa3bc2009a2f9b97cdd43e8b5b7.png",
      "profile_background_tile": false,
      "profile_image_url": "http://pbs.twimg.com/profile_images/877153924637175809/deHwf3Qu_normal.jpg",
      "profile_image_url_https": "https://pbs.twimg.com/profile_images/877153924637175809/deHwf3Qu_normal.jpg",
      "profile_banner_url": "https://pbs.twimg.com/profile_banners/87818409/1491899430",
      "profile_link_color": "005789",
      "profile_sidebar_border_color": "FFFFFF",
      "profile_sidebar_fill_color": "CAE3F3",
      "profile_text_color": "333333",
      "profile_use_background_image": false,
      "has_extended_profile": false,
      "default_profile": false,
      "default_profile_image": false,
      "following": true,
      "follow_request_sent": false,
      "notifications": false,
      "translator_type": "regular"
    },
    "geo": null,
    "coordinates": null,
    "place": null,
    "contributors": null,
    "is_quote_status": false,
    "retweet_count": 1,
    "favorite_count": 3,
    "favorited": false,
    "retweeted": false,
    "possibly_sensitive": false,
    "possibly_sensitive_appealable": false,
    "lang": "en"
  }
 ----------------------------------------------------------------------------------------------------------
I have implemented this in a way that calls from the server using a "setInterval" ever 60 seconds which keeps me within my rate limit:
setInterval(function(){
  getTwitterFeed();
}, 60000);
I then do a web client GET of this data in order to populate my dashboard. I'm currently working with the last five tweets and will probably post how I'm doing that in a later post.

For now. Adieu. I hope you enjoyed reading, feel free to tell me where I'm wrong and how I can improve. you can email me at aiden.g@live.co.uk  or leave me a comment.

Twitter Feed on my dashboard, reloading every sixty seconds

Tuesday, 19 September 2017

National Rail Enquiries Darwin Lite Webservice and Node JS

Getting train times for my local dashboard

Recently I decided I wanted to make a dashboard using Node JS that would provide quick information that I look for regularly. These might be;
  - Share prices I'm interested in
  - Latest football scores and upcoming matches for my football team (Burton Albion)
  - A sub section of my twitter feed

What I definitely wanted was to be able to click onto my dashboard and quickly find out the next trains I can get back to Winchester as I've been commuting to and from London recently and leave for home at differing times. There can occasionally be delays and I'd like to see if there's any delays so I can leave early or later to avoid a rush.

So where to look? The obvious place to try was a quick google of network rail api's which took me to: http://www.nationalrail.co.uk/46391.aspx

The three API options were:
Online Journey Planner - This deals more with planning routes, calculating ticket prices etc. Which since I follow the same route everyday and buy a daily travel card, it doesn't really fit my need.

Knowledgebase - This provides information on engineer disruption, station data such as facilities and service disruption. Whilst the latter is helpful the others were less so, as my stations are always the same and I would be aware of engineer work on the journey to London.

Darwin - In its own words "GB rail industry’s official train running information engine, providing real-time arrival and departure predictions, platform numbers, delay estimates, schedule changes and cancellations." which fits my requirements perfectly!

To use the Darwin SOAP API's you need an authentication token which you get by registering here:

This limits you to 5000 requests / hour which is around 5 million over 28 days. This cap can be removed by contacting infoservices@nationalrail.co.uk though it's extremely unlikely I will hit that!

You can find all the documentation for the API's here:

*NOTE*: You must follow the brand guidelines if you are using the APIs attributing the data to network rail. Brand Guidelines & Logos

So how did I use it?

Great Question!
Now I've not used SOAP much so did the obvious thing and imported the supplied WSDL (v2017-02-02) (https://lite.realtime.nationalrail.co.uk/OpenLDBWS/wsdl.aspx?ver=2017-02-02) into SOAPUI (v5.3.0).

This gave me all the services I can use as well as example XML formats for input to a REST API.



This gives me the HTTP address to make the calls to, the valid XML the service requires and lets me test with the data I will be using.
       <typ:TokenValue> : This is supplied by Network Rail after registration and is required
      <ldb:numRows> : This denotes the number of responses to return.
       <ldb:crs>  : The first of these which is required states the departure station, in my case Waterloo  
        or WAT in network rail world. Full list: http://www.nationalrail.co.uk/stations_destinations/default.aspx 
                          : The second is used for destination station and more can be added for other 
       destination.
      <ldb:timeOffset> : This is how much after the current time you want the search to look for.
      <ldb:timeWindow> : Denotes amount of time to get results for

The output is also very straight forward based on SOAP UI:

Making the request in Node JS

In our node JS application we can achieve the equivalent of this call by using the "request" node module: https://www.npmjs.com/package/request

var request = require("request"); 
var postOptions = {
      url: 'https://lite.realtime.nationalrail.co.uk/OpenLDBWS/ldb10.asmx',
      headers: {
        'Content-Type': 'text/xml'
      },
      body: data/waterlooToWinchester.xml,
      method: 'POST'
    };
    request(postOptions, function cb(err, res, body) {
      if(err) {
        console.error(err);
      } else { 
console.log(body);
}  
Where "data/waterlooToWinchester.xml" is the same XML shown in the SOAPUI request example.

This outputs:
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><GetDepBoardWithDetailsResponse xmlns="http://thalesgroup.com/RTTI/2017-02-02/ldb/"><GetStationBoardResult xmlns:lt6="http://thalesgroup.com/RTTI/2017-02-02/ldb/types" xmlns:lt="http://thalesgroup.com/RTTI/2012-01-13/ldb/types" xmlns:lt4="http://thalesgroup.com/RTTI/2015-11-27/ldb/types" xmlns:lt5="http://thalesgroup.com/RTTI/2016-02-16/ldb/types" xmlns:lt2="http://thalesgroup.com/RTTI/2014-02-20/ldb/types" xmlns:lt3="http://thalesgroup.com/RTTI/2015-05-14/ldb/types"><lt4:generatedAt>2017-09-19T22:44:55.6600574+01:00</lt4:generatedAt><lt4:locationName>London Waterloo</lt4:locationName><lt4:crs>WAT</lt4:crs><lt4:filterLocationName>Winchester</lt4:filterLocationName><lt4:filtercrs>WIN</lt4:filtercrs><lt4:platformAvailable>true</lt4:platformAvailable><lt6:trainServices><lt6:service><lt4:std>23:05</lt4:std><lt4:etd>On time</lt4:etd><lt4:operator>South Western Railway</lt4:operator><lt4:operatorCode>SW</lt4:operatorCode><lt4:serviceType>train</lt4:serviceType><lt4:length>5</lt4:length><lt4:serviceID>0rdRe8ywmDifl5VO1N1Q4Q==</lt4:serviceID><lt5:rsid>SW927700</lt5:rsid><lt5:origin><lt4:location><lt4:locationName>London Waterloo</lt4:locationName><lt4:crs>WAT</lt4:crs></lt4:location></lt5:origin><lt5:destination><lt4:location><lt4:locationName>Poole</lt4:locationName><lt4:crs>POO</lt4:crs></lt4:location></lt5:destination><lt6:subsequentCallingPoints><lt6:callingPointList><lt4:callingPoint><lt4:locationName>Woking</lt4:locationName><lt4:crs>WOK</lt4:crs><lt4:st>23:31</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Basingstoke</lt4:locationName><lt4:crs>BSK</lt4:crs><lt4:st>23:51</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Micheldever</lt4:locationName><lt4:crs>MIC</lt4:crs><lt4:st>00:03</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Winchester</lt4:locationName><lt4:crs>WIN</lt4:crs><lt4:st>00:11</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Eastleigh</lt4:locationName><lt4:crs>ESL</lt4:crs><lt4:st>00:22</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Southampton Airport Parkway</lt4:locationName><lt4:crs>SOA</lt4:crs><lt4:st>00:27</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>St Denys</lt4:locationName><lt4:crs>SDN</lt4:crs><lt4:st>00:32</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Southampton Central</lt4:locationName><lt4:crs>SOU</lt4:crs><lt4:st>00:37</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Totton</lt4:locationName><lt4:crs>TTN</lt4:crs><lt4:st>00:43</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Brockenhurst</lt4:locationName><lt4:crs>BCU</lt4:crs><lt4:st>00:54</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>New Milton</lt4:locationName><lt4:crs>NWM</lt4:crs><lt4:st>01:02</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Christchurch</lt4:locationName><lt4:crs>CHR</lt4:crs><lt4:st>01:09</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Pokesdown</lt4:locationName><lt4:crs>POK</lt4:crs><lt4:st>01:13</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Bournemouth</lt4:locationName><lt4:crs>BMH</lt4:crs><lt4:st>01:17</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Branksome</lt4:locationName><lt4:crs>BSM</lt4:crs><lt4:st>01:23</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Parkstone (Dorset)</lt4:locationName><lt4:crs>PKS</lt4:crs><lt4:st>01:26</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint><lt4:callingPoint><lt4:locationName>Poole</lt4:locationName><lt4:crs>POO</lt4:crs><lt4:st>01:30</lt4:st><lt4:et>On time</lt4:et><lt4:length>5</lt4:length></lt4:callingPoint></lt6:callingPointList></lt6:subsequentCallingPoints></lt6:service></lt6:trainServices></GetStationBoardResult></GetDepBoardWithDetailsResponse></soap:Body></soap:Envelope>
This isn't much good to me at least, as I prefer to use JSON to access the data at the web client side. To do this there is a neat little node module called "xml2js": https://www.npmjs.com/package/xml2js

The below snippet parses the xml string and removes the prefixes from the xml tags to make it more readable and accessible as JSON.

var parseString = require('xml2js').parseString;
var stripPrefix = require('xml2js').processors.stripPrefix;
parseString(body, { tagNameProcessors: [stripPrefix] }, function (err, result) {
            if(err){
              console.log(err);
            }else{
              var currentJourney = result.Envelope.Body[0].GetDepBoardWithDetailsResponse[0].GetStationBoardResult[0].trainServices[0].service[0];
                  var train = {
                    "trainTime": currentJourney.std[0],
                    "status": currentJourney.etd[0],
                    "departingFrom": currentJourney.origin[0].location[0].locationName[0],
                    "goingTo": currentJourney.destination[0].location[0].locationName[0]
                  };
}, function afterJourneysProcessed(err) {
                if(err) {
                  console.log(err);
                }else{
                  return(train);
                }
});
At the client side we can handle this "train" variable by parsing the JSON and then setting the html based on the JSON input.

dashboard.html
<p id="nexttrains1"></p>
dashboard.js

var example = JSON.parse(this.responseText);
        //train 1
        var trainOutput1 = example[0].departingFrom + " to " + example[0].goingTo + "<br>";
        trainOutput1 = trainOutput1 + "<b>Departing at:</b> " + example[0].trainTime + " <b>Current Status:</b> " + example[0].status;
        document.getElementById("nexttrains1").innerHTML = trainOutput1;
This returns a single train journey. In my own code I also included an array called "calling points" and returned five journeys. This is quite a bit more code and a little too much for a single blog (since I've already added a lot already) but I can pass it over on request. Email me if you'd like to see the full code with promises and client to server code!

What does it look like? 


Anything you would do differently?
Any questions?

Feel free to email or comment and I'll gladly get back to you!

Thursday, 17 August 2017

Selenium Tips - Node JS

So I've recently started making a few web user interface tests that can't be manually achieved with API calls. I was looking forward to practising my Node JS so thought I'd make it using that framework.

Here's a few little functions and commands to both get you going or help me remember what I've done:

Firstly, Selenium is a web browser automation software that can be run on the browser, in an application such as node or as an IDE. find out more HERE

Secondly, Node JS my desired programming service of the day is an open source platform built on a javascript runtime. There's a great youtube introductory video HERE

So... Getting Started. So I had installed node and created a dedicated directory.
I navigated to my new directory and installed selenium-webdriver:
 npm install selenium-webdriver
I then created an application:
echo "" > myNewApp.js
In the application I referenced selenium-webdriver and then used selenium builder to create a new browser driver like so:
const selenium = require('selenium-webdriver'),
    By = selenium.By,
    until = selenium.until;

var driver = new selenium.Builder()
    .forBrowser('chrome')
    .build();

var driver = new selenium.Builder()
    .forBrowser(‘firefox’)
    .build();
You'll either want different variable names for your different browsers or to only call one at a time!
From here on there are a whole bunch of commands we can run using driver.Command you can see a  list of some of the commands HERE.

The next thing to do is to perform a GET function on the webpage you want to navigate to in your automated test.
driver.get('www.myUrl.com').then(function(){
   console.log('Navigating to my page');

});
From this page we can perform a whole bunch of commands:
Populate an object:
driver.findElement(By.name(‘red’)).sendKeys(‘ABC’);
Clear a populated object:
driver.findElement(By.name(‘red’)).clear();
Make a hidden object visible:
driver.executeScript("document.getElementsByName(‘red’)[0].setAttribute('type', 'text');");
Using the drivers .then function will let you wait for something to occur and then perform the next action, such as wait for 2 seconds:
.then(function(){
  driver.sleep(2000);
});
You might want to click a button, all you need to know is the button class name. One way of doing this is to "inspect" the webpage from the browser.
Click:
driver.findElement(By.className('btn btn-lg')).click(); 
To quit the browser:
driver.quit(); 
The one thing I wasn't able to do was make a webpage stop loading to allow me to process a page that didn't stop. Let me know if you've done this before! I'd love to hear how you did it. 

Saturday, 5 August 2017

A Beginners Guide to the Allotment

Louise and I have had the allotment for just over a year now and I thought it would be a good idea to get some thoughts down on our "Lessons Learnt" and some general advice for starting your own allotment.

Getting Started:
  1. Wait times for allotments can be very long! Up to 6 months in some councils as allotments need to be vacated before they can be reallocated.
  2. Allotments are measured in rods where 1 rod is around 5 meters
  3. Cost varies from council to council
  4. The first few tools you'll need are some pots, a spade, a fork and a watering can
  5. You will need a compost bin, pick an out of the way location that is relatively flat
  6. Start small, no need to dig the whole thing over at once.
  7. Ask neighbours for advice, most will be happy to help
  8. You can buy old tools very cheap at weekend markets, and local allotment organisations often sell seeds and compost, your local farmer will sell horse manure very cheap too
The What's What:
Some things look weird can you may find yourself wondering what other people are doing and why they have done them:
  1. Raised Beds - These are wooden borders around a small patch of land, they make it harder for slugs to eat your seeds.
  2. Fire Pits - Not all allotments allow them, and some only allow them between certain times. But here you can burn your weeds (like bind weed). The ash also makes great fertiliser!
  3. Fruit Cages - These keep the birds off the berries and trees
  4. Nets -  There will be nets and tunnels over seedlings, again to stop the birds. Some nets are specially designed to keep off frost, snow and bugs but still allow rain and sunlight.
  5. Tarp - This is usually put over a particularly weedy patch of land, it blocks out sunlight and kills the weeds at the roots. Normally left over the year ready for the new year.
Chemical Romance:
Here's some of the do and do nots around chemicals
  1. Slug Killer - in small doses keeps slugs at bay. However, pellets stay inside the slugs and the poison can have adverse affects on hedgehogs who can die from poisoning as well. There is also the case whereby if slug levels become too low there will be less millipedes, frogs and hedgehogs on your allotment meaning future slug populations will thrive without natural predators! Alternatives include; slug traps, putting them in the compost bin, raised beds, salt walls around plants.
  2. Weed Killer - I'm not adverse to weed killer itself but in the wind it can blow to other allotments killing other peoples vegetables. This is a big no no for obvious reasons. 
  3. Bug Killer - There are lots of home made formulas for killing white fly and aphids but some store bought products work just well.
Lessons Learnt:
We've all made mistakes in our lives, here are some of ours:
  1. Make sure you know what fruit/veg is what! We've pulled out strawberries we thought were weeds and grown raspberry bushes we thought was cauliflower. There's lots of apps for knowing what is what and books too!
  2. Plan what you will grow and when. Veg needs to be seeding pretty early in the year and inside your house is a perfect place whilst you get the allotment ready. Plants grow better in the greenhouse but the house can be good.
  3. Find a friend to agree to water your allotment whenever you're away. Alternatively fellow allotmenteers will sometimes offer, hence why we don't spray our neighbours allotments with weed killer! This is mainly important at the seed stage, less so when they're out in the wild.
  4. If you can't find someone, then there's ways of watering your seeds without human intervention. I've mentioned these before here: and we personally settled for this.
  5. Be careful when digging grassy bits of land, creatures great and small live there. Mice, frogs, hedgehogs etc. Killing a frog by accident is the worst! :(
  6. Composting can be a bit of an art. There's green and brown types of waste that can go in. Green is veg, leftover food and some weeds. Browns are dry leaves, cardboard from your house, tea bags and coffee grounds. The eden project do a good introduction here and a comprehensive video guide here.
That's all I have time for in this sitting! But I'll no doubt be back with a Part 2!

Do you have your own tips for new allotment holders? Are you having any particular problems on your own allotment? 

Leave a comment below to get in touch :)