Showing posts with label javascript. Show all posts
Showing posts with label javascript. Show all posts

Wednesday, 27 February 2019

Football Simulation Engine v2.1.0, v2.1.2 and v2.2.1

The football simulation engine node module has been available for just over a year now and has a couple of hundred downloads, and an interaction of a fair few people via GitHub. Part of that interaction has led to requests for changes to be made to the modules functions to either make the game easier to control or alter. 

Whilst making the changes other issues were thrown up such as the ball not continuing to move with the players when they were in possession, and in other cases the ball remaining in the players control after it had been kicked. The new versions - specifically version 2.2.1 - resolves these issues and provides a much better and more reliable output.

Refactoring 

The first big change was the removal of the async module. Two reasons for this were that a) it didn’t need it, b) I’ve grown a distaste for dependencies in my NodeJS programs. On top of this there were changes to allow for the latest node version to be used which included using let, const instead of var which good practice dictates and the general removal of promises which were also not necessary. 

General input from https://github.com/dashersw has been instrumental which has included improving error throwing as well as removing repetition. As such the code has been regularly reduced or made easier to understand with game examples now available.

Testing 


Part of my own journey has been to begin looking at Test Driven Development (TDD). Unfortunately, I could only apply it in this module retrospectively but will be adding test cases for all new functionality before I code it in future. 

For now, I have added testing for the three main exported functions and testing of validation of input. This is because it is where users are most likely to get stuck. My new goal is also to raise tests around whatever issues come through. One example is https://github.com/GallagherAiden/footballSimulationEngine/issues/33 where tests were added to validate the “ClosestPlayer” functions.

Adding tests has improved the game, because it has caught a number of interesting ‘perks’ of the code written where tests that should have passed actually failed. Which is good, as it shows TDD in action. 

Finally, part of the testing now includes End to End testing through Match Simulation of varying team ratings which shows expected results for teams of different skills.

New Functionality


After a review of the games being played there were a few obvious changes that could be made. Allowing more tackleswas one such whereby players rarely were able to take the ball off each other before, this has now changed and with it comes an improved risk of players committing fouls. 

The greatest new functionality was to stop the ball moving instantaneously, instead the ball moves over a period of iterations. This includes an x, y and z movement which factors in the height of the ball against the height of another player. This has allowed even greater play by introducing interception of the ball as it moves

Another big new function to the simulation was in the calculation of offside and the beginnings of some simple player marking. Other improvements have been possible with the ball movement change which includes; direction of player movementso players attack in the right direction, and better passing to stop players constantly passing to the keeper.

Finally, the way shots were taken, calculated and checked for so that own goals are awarded to the correct teams and shots occur over time. This has also led to an improvement of the ability for keepers to save shots as the ball enters their area.

Finally, a player fitness parameter has been added however this has no impact on the game currently and is merely additional information for people using the simulation engine as a manger game. This also includes an action parameter for each player which allows the move performed by each player determined by the simulator to be overridden.

Fixes


There have been a number of little fixes such as elimination of incorrect reporting of player possession of the ball. Changes have been made so that the closest player to ball or a position now gives a more accurate output.

Players now move with the ball at their feetwhich was quite a major bug in previous versions.



The changes made have had a massive impact on game play which you can see on the video I created for the latest version.



There are other changes coming in version 3 which looks to make the iteration json read better and make it more accessible. Other possibilities will be the removal of players from game play e.g. by red or yellow cards and the ability for team rating and other skills and fitness to have a greater impact on individual player performance.

I would love to receive feedback and requests for improvement which can be added here: https://github.com/GallagherAiden/footballSimulationEngine/issues
Alternatively you can email me here.

Happy footballing and thank you for downloading and using the module. 

Wednesday, 22 November 2017

Simulating Football / Soccer Results

I've recently started to look for a football simulation engine (soccer over in the USA) that can be used  to begin a football manager game. I realised quickly this is something lots of people try but don't necessarily finish or keep on top of:
- https://github.com/d-nation/soccer-sim-engine
https://github.com/atas76/SimpleFootie
and there's plenty of online forums asking how you might make a simple match simulator, so I thought why not just give it a go and see what I come up with.

My code language of choice is Javascript and I'll be using NodeJS throughout.

Attempt 1 - Use Team Rating to generate prediction

This uses a function that generates a random number
function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}
we can then feed minimum and maximum numbers to get a random number between the two.
This attempt uses the team rating to determine different maximums, so that better rated teams can get a higher score but it isn't impossible for the lower teams to win.
function getHighestLikely(teamRating) {
    if (teamRating < 100 && teamRating > 80) {
        return 20;
    } else if (teamRating < 79 && teamRating > 60) {
        return 8;
    } else if (teamRating < 59 && teamRating > 30) {
        return 2;
    } else if (teamRating < 29 && teamRating > 0) {
        return 1;
    }
}
we can then run the following where the team ratings can be passed in by the user:
var team1rating = 73;
var team2rating = 45;
var homeTeamScore = getRandomNumber(0, getHighestLikely(team1rating)); 
var awayTeamScore = getRandomNumber(0, getHighestLikely(team2rating));
console.log("Home: " + homeTeamScore + " - " + awayTeamScore + " : Away");

Attempt 2 - Random Event, Random Team Member and Player Rating Comparison

I then looked at how we can utilise some random shooting events alongside a random team member and a comparison of two players ratings.

For this we will need:
- an events variable for goal scoring events
- readFile function (will show below)
- two JSON files with players and player ratings
function readFile(filePath) {
    return new Promise(function (resolve, reject) {
        fs.readFile(filePath, 'utf8', function (err, data) {
            if (err) {
                reject(err);
            } else {
                data = JSON.parse(data);
                resolve(data);
            }
        })
    });
}
We use the above function to read in two json files with player ratings, in this example I've put a lot of information that I may or may not use in the future:
{
  "name": "Brewers",
  "rating": "89",
  "players": [{
      "name": "Bill Gallagher",
      "position": "GK",
      "rating": "75",
      "skill": {
        "passing": "78",
        "shooting": "12",
        "saving": "75",
        "penalty_taking": "43"
      }
    },
    {
      "name": "Fred Gallagher",
      "position": "LB",
      "rating": "90",
      "skill": {
        "passing": "83",
        "shooting": "40",
        "tackling": "32",
        "penalty_taking": "53"
      }
    },
    {
      "name": "George Gallagher",
      "position": "CB",
      "rating": "84",
      "skill": {
        "passing": "78",
        "shooting": "37",
        "tackling": "21",
        "penalty_taking": "66"
      }
    },
    {
      "name": "Jim Gallagher",
      "position": "CB",
      "rating": "75",
      "skill": {
        "passing": "33",
        "shooting": "76",
        "tackling": "76",
        "penalty_taking": "21"
      }
    },
    {
      "name": "Sid Gallagher",
      "position": "RB",
      "rating": "82",
      "skill": {
        "passing": "66",
        "shooting": "65",
        "tackling": "81",
        "penalty_taking": "88"
      }
    },
    {
      "name": "Gregory Gallagher",
      "position": "LM",
      "rating": "87",
      "skill": {
        "passing": "51",
        "shooting": "88",
        "tackling": "81",
        "penalty_taking": "94"
      }
    },
    {
      "name": "Arthur Gallagher",
      "position": "CM",
      "rating": "41",
      "skill": {
        "passing": "33",
        "shooting": "66",
        "tackling": "55",
        "penalty_taking": "1"
      }
    },
    {
      "name": "Cameron Gallagher",
      "position": "CM",
      "rating": "99",
      "skill": {
        "passing": "88",
        "shooting": "95",
        "tackling": "91",
        "penalty_taking": "62"
      }
    },
    {
      "name": "Tanisha Gallagher",
      "position": "RM",
      "rating": "79",
      "skill": {
        "passing": "56",
        "shooting": "79",
        "tackling": "74",
        "penalty_taking": "32"
      }
    },
    {
      "name": "Aiden Gallagher",
      "position": "ST",
      "rating": "75",
      "skill": {
        "passing": "83",
        "shooting": "88",
        "tackling": "59",
        "penalty_taking": "45"
      }
    },
    {
      "name": "Louise Peverley",
      "position": "ST",
      "rating": "88",
      "skill": {
        "passing": "73",
        "shooting": "61",
        "tackling": "44",
        "penalty_taking": "66"
      }
    }
  ],
  "manager": "Aiden",
  "formation": [4,4,2]
}
the two json files differed only slightly.
I will also require a variable called "events" : var events = ["shot", "penalty"];
function playMatch(team1Config, team2Config) {
    var matchEventsNo = getRandomNumber(0, 10);
    console.log("Total Events in the Match: " + matchEventsNo);
    readFile(team1Config).then(function (team1) {
        readFile(team2Config).then(function (team2) {
            while (matchEventsNo != 0) {
                console.log("Event: " + matchEventsNo);
                var eventTeam = getRandomNumber(1, 2);
                var eventPlayerHome = team1.players[getRandomNumber(0, 10)];
                var eventPlayerAway = team2.players[getRandomNumber(0, 10)];
                if (eventTeam === 1) {
                    console.log("Event Team: " + team1.name);
                } else if (eventTeam === 2) {
                    console.log("Event Team: " + team2.name);
                }
                var thisEvent = events[getRandomNumber(0, 1)];
                console.log("Event Type: " + thisEvent);
                if (thisEvent === "penalty") {
                    if (eventTeam === 1) {
                        console.log("Player: " + eventPlayerHome.name + "(" + eventPlayerHome.position + ") rating: " + eventPlayerHome.rating);
                        console.log("Player: " + team2.players[0].name + "(" + team2.players[0].position + ") rating: " + team2.players[0].rating);
                        if (eventPlayerHome.rating > team2.players[0].rating) {
                            homeScore++;
                        }
                    } else if (eventTeam === 2) {
                        console.log("Player: " + eventPlayerAway.name + "(" + eventPlayerAway.position + ") rating: " + eventPlayerAway.rating);
                        console.log("Player: " + team1.players[0].name + "(" + team1.players[0].position + ") rating: " + team1.players[0].rating);
                        if (eventPlayerAway.rating > team1.players[0].rating) {
                            awayScore++;
                        }
                    }
                } else if (thisEvent === "shot") {
                    if (eventTeam === 1) {
                        console.log("Player: " + eventPlayerHome.name + "(" + eventPlayerHome.position + ") rating: " + eventPlayerHome.rating);
                        console.log("Player: " + team2.players[0].name + "(" + team2.players[0].position + ") rating: " + team2.players[0].rating);
                        if (eventPlayerHome.rating > team2.players[0].rating) {
                            homeScore++;
                        }
                    } else if (eventTeam === 2) {
                        console.log("Player: " + eventPlayerAway.name + "(" + eventPlayerAway.position + ") rating: " + eventPlayerAway.rating);
                        console.log("Player: " + team1.players[0].name + "(" + team1.players[0].position + ") rating: " + team1.players[0].rating);
                        if (eventPlayerAway.rating > team1.players[0].rating) {
                            awayScore++;
                        }
                    }
                }
                matchEventsNo--;
                console.log(team1.name + " " + homeScore + " - " + awayScore + " " + team2.name);
            }
        });
    });
}

function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}
This created up to ten random events per "Match". For each match it would then decide which of two events had occurred. Either a penalty or a shot. It would then decide which of the two teams (randomly again) has incurred the event. We then channel a series of if statements to determine if there was a goal by comparing the player who took the shot/penalty (randomly chosen from the team) against the opposition goalkeepers ability.

We can run this using:
playMatch("teams/team1.json", "teams/team2.json").then(function(){ 
});

This gives us a limited number of goals (10 max) but can be increased by saying a minimum of X events and a maximum of (infinite) events to increase goals that can be scored.

Random "Plays" made of up to 10 random events.

Whilst this was good and worked, I thought the best way to boost this method of generating football events and therefore goals was to create "plays" which would consist of 10 random events. These would begin by an event starter such as a throw in, corner, goal kick, free kick or a penalty.

All of these are events that "start" play from a stoppage. After this has been determined a continue of the event is decided as either pass, cross or shoot.  A play is only finished being generate once a "shoot" token has been given.

Once the 10 plays have been decided with their potentially infinite events, there is a formula run to see if a goal is scored. First we decide which team is doing each of the 10 events (an array of 10, each number in the array either being 0 or 1 to determine if the home or away team are performing the "play").

We then compare the players skill rating for the given event against a random number between 0 and 100. Thus higher rated players are more likely to be successful in whatever event they are doing i.e. passing. At any stage a pass, cross, shot or penalty can miss thus ending the play and meaning no goal will be scored.

var fs = require("fs");
var async = require("async");
var teamName;
var userName;
var eventStart = ["throwin", "corner", "goalkick", "freekick", "penalty"];
var events = ["pass", "cross", "shoot"];
var homeScore = 0;
var awayScore = 0;
var play = [];
var eventTeams = [];

playMatch("teams/team1.json", "teams/team2.json");
resetVars();

function playMatch(team1Config, team2Config) {
    eventTeam().then(function () {
        readFile(team1Config).then(function (team1) {
            readFile(team2Config).then(function (team2) {
                async.eachSeries(eventTeams, function eachTeam(thisTeam, thisTeamCallback) {
                    console.log("--------------------");
                    console.log("Starting Play");
                    if (thisTeam === 0) {
                        generatePlay().then(function () {
                            goalScored(team1, team2, team1).then(function (score) {
                                if (score === 1) {
                                    console.log("Team 1 scored");
                                    homeScore++;
                                    play = [];
                                    thisTeamCallback();
                                } else {
                                    console.log("Team 1 missed");
                                    play = [];
                                    thisTeamCallback();
                                }
                            });
                        });
                    } else {
                        generatePlay().then(function () {
                            goalScored(team1, team2, team2).then(function (score) {
                                if (score === 1) {
                                    console.log("Team 2 scored");
                                    awayScore++;
                                    play = [];
                                    thisTeamCallback();
                                } else {
                                    console.log("Team 2 missed");
                                    play = [];
                                    thisTeamCallback();
                                }
                            });
                        });
                    }
                }, function afterAllTppUserOrgs() {
                    console.log(team1.name + " " + homeScore + " - " + awayScore + " " + team2.name);
                });
            });
        });
    });
}

function eventTeam() {
    return new Promise(function (resolve, reject) {
        var playTotal = getRandomNumber(1, 10);
        console.log("Total plays in the Match: " + playTotal);
        while (playTotal !== 0) {
            eventTeams.push(getRandomNumber(0, 1));
            playTotal--;
            if (playTotal === 0) {
                resolve();
            }
        }
    });
}

function goalScored(team1, team2, whichTeam) {
    return new Promise(function (resolve, reject) {
        var score = 0;
        async.eachSeries(play, function eachEvent(playEvent, playEventCallback) {
            console.log(playEvent);
            if (score === -1) {
                playEventCallback();
            } else {
                if (playEvent === "throwin" || playEvent === "corner" || playEvent === "goalkick" || playEvent === "freekick" || playEvent === "pass" || playEvent === "cross") {
                    if (whichTeam.players[getRandomNumber(1, 10)].skill.passing > getRandomNumber(0, 100)) {
                        playEventCallback();
                    } else {
                        score = -1;
                        resolve(score);
                    }
                } else if (playEvent === "penalty" || playEvent === "shoot") {
                    if (whichTeam.players[getRandomNumber(1, 10)].skill.shooting > getRandomNumber(0, 100)) {
                        console.log("goal scored");
                        score++;
                        resolve(score);
                    } else {
                        console.log("shot missed");
                        resolve(score);
                    }
                }
            }
        }, function afterAllEvents() {
            resolve(score);
        });
    });
}

function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}

function generatePlay() {
    return new Promise(function (resolve, reject) {
        var startEvent = eventStart[getRandomNumber(0, 4)];
        play.push(startEvent);
        if (startEvent === "penalty") {
            resolve(play);
        } else {
            newEventInPlay(function () {
                resolve(play);
            });
        }
    });
}

function newEventInPlay(callback) {
    newEvent = events[getRandomNumber(0, 2)];
    play.push(newEvent);
    if (newEvent === "shoot") {
        callback();
    } else {
        newEventInPlay(callback);
    }
}

function resetVars() {
    homeScore = 0;
    awayScore = 0;
}

function readFile(filePath) {
    return new Promise(function (resolve, reject) {
        fs.readFile(filePath, 'utf8', function (err, data) {
            if (err) {
                reject(err);
            } else {
                data = JSON.parse(data);
                resolve(data);
            }
        })
    });

Added tackling and saving to the formula

My final thought was how I could incorporate the defensive teams statistics to make it more of a two team game. To do this, when a player misses a pass or cross etc. We see if they were tackled or intercepted after the failure.
If yes, the particular play ends with no further actions. If not, the play continues. When a player shoots successfully (the rating is higher than the random number generated between 0 and 100) we see if the keeper was able to save it. To alleviate the bias this gives against scoring, we play many more events. In this example, 30.
var fs = require("fs");
var async = require("async");
var eventStart = ["throwin", "corner", "goalkick", "freekick", "penalty"];
var events = ["pass", "cross", "shoot"];
var homeScore = 0;
var awayScore = 0;
var play = [];
var eventTeams = [];

playMatch("teams/team1.json", "teams/team2.json");

function playMatch(team1Config, team2Config) {
        eventTeam().then(function () {
            readFile(team1Config).then(function (team1) {
                readFile(team2Config).then(function (team2) {
                    console.log("Todays Match is: " + team1.name + " vs " + team2.name);
                    async.eachSeries(eventTeams, function eachTeam(thisTeam, thisTeamCallback) {
                        //console.log("--------------------");
                        //console.log("Starting Play");
                        if (thisTeam === 0) {
                            generatePlay().then(function () {
                                goalScored(team1, team2).then(function (score) {
                                    if (score === 1) {
                                        //console.log(team1.name + " scored");
                                        homeScore++;
                                        play = [];
                                        thisTeamCallback();
                                    } else {
                                        //console.log(team1.name + " missed");
                                        play = [];
                                        thisTeamCallback();
                                    }
                                });
                            });
                        } else {
                            generatePlay().then(function () {
                                goalScored(team2, team1).then(function (score) {
                                    if (score === 1) {
                                        //console.log(team2.name + " scored");
                                        awayScore++;
                                        play = [];
                                        thisTeamCallback();
                                    } else {
                                        //console.log(team2.name + " missed");
                                        play = [];
                                        thisTeamCallback();
                                    }
                                });
                            });
                        }
                    }, function afterAllTppUserOrgs() {
                        console.log(team1.name + " " + homeScore + " - " + awayScore + " " + team2.name);
                        resetVars();
                        resolve();
                    });
                });
            });
        });
}

function eventTeam() {
    return new Promise(function (resolve, reject) {
        var playTotal = getRandomNumber(1, 30);
        //console.log("Total plays in the Match: " + playTotal);
        while (playTotal !== 0) {
            eventTeams.push(getRandomNumber(0, 1));
            playTotal--;
            if (playTotal === 0) {
                resolve();
            }
        }
    });
}

function goalScored(whichTeam, oppTeam) {
    return new Promise(function (resolve, reject) {
        var score = 0;
        async.eachSeries(play, function eachEvent(playEvent, playEventCallback) {
            var thisPlayer = whichTeam.players[getRandomNumber(1, 10)];
            var oppPlayer = oppTeam.players[getRandomNumber(0, 10)];
            //console.log(thisPlayer.name + " (" + thisPlayer.position + ")");
            //console.log(playEvent);
            if (score === -1) {
                playEventCallback();
            } else {
                if (playEvent === "throwin" || playEvent === "corner" || playEvent === "goalkick" || playEvent === "freekick" || playEvent === "pass" || playEvent === "cross") {
                    if (thisPlayer.skill.passing > getRandomNumber(0, 100)) {
                        playEventCallback();
                    } else {
                        //console.log("Retain play?");
                        //console.log("Challenge made by: " + oppPlayer.name + " (" + oppPlayer.position + ")");
                        if (oppPlayer.skill.tackling > getRandomNumber(0, 100)) {
                            //console.log("tackled. posession lost");
                            score = -1;
                            resolve(score);
                        } else {
                            //console.log("retained possession. Continue play");
                            playEventCallback();
                        }
                    }
                } else if (playEvent === "penalty" || playEvent === "shoot") {
                    if (thisPlayer.skill.shooting > getRandomNumber(0, 100)) {
                        if (oppTeam.players[0].skill.saving > getRandomNumber(0, 100)) {
                            //console.log("shot saved by " + oppTeam.players[0].name + " (" + oppTeam.players[0].position + ")");
                            resolve(score);
                        } else {
                            //console.log("goal conceeded by " + oppTeam.players[0].name + " (" + oppTeam.players[0].position + ")");
                            //console.log("goal scored by " + thisPlayer.name + " (" + thisPlayer.position + ")");
                            score++;
                            resolve(score);
                        }
                    } else {
                        //console.log("shot missed");
                        resolve(score);
                    }
                }
            }
        }, function afterAllEvents() {
            resolve(score);
        });
    });
}

function getRandomNumber(min, max) {
    var random = Math.floor(Math.random() * (max - min + 1)) + min;
    return random;
}

function generatePlay() {
    return new Promise(function (resolve, reject) {
        var startEvent = eventStart[getRandomNumber(0, 4)];
        play.push(startEvent);
        if (startEvent === "penalty") {
            resolve(play);
        } else {
            newEventInPlay(function () {
                resolve(play);
            });
        }
    });
}

function newEventInPlay(callback) {
    newEvent = events[getRandomNumber(0, 2)];
    play.push(newEvent);
    if (newEvent === "shoot") {
        callback();
    } else {
        newEventInPlay(callback);
    }
}

function resetVars() {
    homeScore = 0;
    awayScore = 0;
    play = [];
    eventTeams = [];
}

function readFile(filePath) {
    return new Promise(function (resolve, reject) {
        fs.readFile(filePath, 'utf8', function (err, data) {
            if (err) {
                reject(err);
            } else {
                data = JSON.parse(data);
                resolve(data);
            }
        })
    });
}

What next?

The next step for this project would require much more intelligent movement, ratings, and event play for the user. I'm planning on developing this over the next few months and with any luck will have something to open source by the middle of next year.
However, this gives quite a good amount of data and events already. At the bottom I will show the output we see using this bit of code. However, if you have used, or adapted this code. Feel free to get in touch and let me know how it goes by email.
Good Luck!
Output:
Total plays in the Match: 22
Todays Match is: Brewers vs Arsenal
--------------------
Starting Play
Jim Boden (CB)
penalty
penalty on target by Jim Boden (CB)
goal conceeded by Bill Gallagher (GK)
Arsenal scored
--------------------
Starting Play
Gregory Gallagher (LM)
throwin
Retain play?
Challenge made by: Sid Boden (RB)
tackled. posession lost
Brewers missed
--------------------
Starting Play
Louise Boden (ST)
throwin
Retain play?
Challenge made by: Bill Gallagher (GK)
retained possession. Continue play
Arthur Boden (CM)
cross
Retain play?
Challenge made by: Cameron Gallagher (CM)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Gregory Boden (LM)
penalty
penalty on target by Gregory Boden (LM)
penalty saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Aiden Boden (ST)
penalty
penalty on target by Aiden Boden (ST)
goal conceeded by Bill Gallagher (GK)
Arsenal scored
--------------------
Starting Play
George Boden (CB)
goalkick
Retain play?
Challenge made by: Louise Peverley (ST)
retained possession. Continue play
Sid Boden (RB)
shoot
goal conceeded by Bill Gallagher (GK)
goal scored by Sid Boden (RB)
Arsenal scored
--------------------
Starting Play
Louise Boden (ST)
corner
Retain play?
Challenge made by: Aiden Gallagher (ST)
retained possession. Continue play
Tanisha Boden (RM)
shoot
shot saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Gregory Boden (LM)
freekick
Gregory Boden (LM)
cross
Cameron Boden (CM)
cross
Gregory Boden (LM)
cross
Sid Boden (RB)
pass
Arthur Boden (CM)
shoot
goal conceeded by Bill Gallagher (GK)
goal scored by Arthur Boden (CM)
Arsenal scored
--------------------
Starting Play
Jim Boden (CB)
goalkick
Retain play?
Challenge made by: Arthur Gallagher (CM)
retained possession. Continue play
George Boden (CB)
pass
Tanisha Boden (RM)
cross
Retain play?
Challenge made by: Cameron Gallagher (CM)
retained possession. Continue play
Aiden Boden (ST)
pass
Gregory Boden (LM)
pass
Fred Boden (LB)
pass
Retain play?
Challenge made by: Cameron Gallagher (CM)
retained possession. Continue play
Sid Boden (RB)
pass
Arthur Boden (CM)
cross
Retain play?
Challenge made by: Bill Gallagher (GK)
retained possession. Continue play
Jim Boden (CB)
cross
Retain play?
Challenge made by: Jim Gallagher (CB)
retained possession. Continue play
Gregory Boden (LM)
shoot
goal conceeded by Bill Gallagher (GK)
goal scored by Gregory Boden (LM)
Arsenal scored
--------------------
Starting Play
Cameron Gallagher (CM)
throwin
Gregory Gallagher (LM)
cross
Retain play?
Challenge made by: Tanisha Boden (RM)
tackled. posession lost
Brewers missed
--------------------
Starting Play
Louise Boden (ST)
goalkick
Cameron Boden (CM)
pass
George Boden (CB)
pass
Retain play?
Challenge made by: Louise Peverley (ST)
retained possession. Continue play
Fred Boden (LB)
shoot
shot missed
Arsenal missed
--------------------
Starting Play
Jim Boden (CB)
freekick
Jim Boden (CB)
cross
Retain play?
Challenge made by: Aiden Gallagher (ST)
retained possession. Continue play
Tanisha Boden (RM)
pass
Retain play?
Challenge made by: Tanisha Gallagher (RM)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Aiden Boden (ST)
corner
George Boden (CB)
cross
Louise Boden (ST)
cross
Retain play?
Challenge made by: George Gallagher (CB)
retained possession. Continue play
Arthur Boden (CM)
shoot
shot saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Sid Gallagher (RB)
corner
Sid Gallagher (RB)
shoot
goal conceeded by Bill Boden (GK)
goal scored by Sid Gallagher (RB)
Brewers scored
--------------------
Starting Play
George Gallagher (CB)
freekick
Gregory Gallagher (LM)
cross
Retain play?
Challenge made by: Louise Boden (ST)
tackled. posession lost
Brewers missed
--------------------
Starting Play
Sid Gallagher (RB)
penalty
penalty on target by Sid Gallagher (RB)
goal conceeded by Bill Boden (GK)
Brewers scored
--------------------
Starting Play
Tanisha Boden (RM)
corner
Sid Boden (RB)
cross
Arthur Boden (CM)
cross
Retain play?
Challenge made by: Jim Gallagher (CB)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Arthur Boden (CM)
goalkick
Retain play?
Challenge made by: Tanisha Gallagher (RM)
retained possession. Continue play
Aiden Boden (ST)
shoot
shot saved by Bill Gallagher (GK)
Arsenal missed
--------------------
Starting Play
Arthur Boden (CM)
corner
Retain play?
Challenge made by: Louise Peverley (ST)
tackled. posession lost
Arsenal missed
--------------------
Starting Play
Tanisha Gallagher (RM)
corner
Sid Gallagher (RB)
shoot
shot saved by Bill Boden (GK)
Brewers missed
--------------------
Starting Play
Arthur Boden (CM)
corner
Tanisha Boden (RM)
pass
George Boden (CB)
shoot
shot missed
Arsenal missed
--------------------
Starting Play
Sid Boden (RB)
throwin
Tanisha Boden (RM)
shoot
shot missed
Arsenal missed
Brewers 2 - 5 Arsenal

Tuesday, 26 September 2017

Using twitters user timeline API in Node JS

Having continued to create my dashboard that allows me to access information thats useful to me in a single, easy to access location, I began to think about how I could get my twitter timeline to display on screen (messages that I post and messages people I follow post).

Normally I like to try to just do a curl GET of the webpage I'm looking to use but twitter requires an oauth key to do even a get on your own profile. *sigh*.

Since I've started developing Twitter have changed their entire web based content to a flash new website, a few new APIs and a new "feel".
New Twitter Site

The first thing we need to become a twitter developer is to create an application on https://apps.twitter.com which requires a login and for your account to have a linked mobile number. You'll need to give the application a name and a website address where the application will later ve accessible (this doesn't have to be real and can be changed later).

You will then be given application settings information complete with access token, and security keys, specifically within the "Keys and access tokens" tab.
Application Settings
-----------------------------------------------------
What I specifically wanted to do in my dashboard was display a few of the latest tweets from my home timeline which was surprisingly difficult to figure out as most use cases seem to be around mass production of tweets and making "bots" and a lot of the documentation for node module implementation on node js didn't really describe what was happening.

It's worth noting that when thinking about how I wanted to achieve this I had to consider twitters rate limiting for per access token usage. Which comes to 15 calls per application/access token per 15 minute window, meaning realistically we could make one call a minute.

Initially I just wanted to get set up and the easiest way to do so seemed to be with an existing node module called (wait for it...) twitter npm. https://www.npmjs.com/package/twitter. Following their instructions I took a plain old Node Js application and added the following code using the oauth tokens we get in our application settings:
var Twitter = require('twitter');
var client = new Twitter({
      consumer_key: 'XXX',
      consumer_secret: 'XXX',
      access_token_key: 'XXX',
      access_token_secret: 'XXX'
    });
At this point there are two options for receiving tweets from our timeline and this is to use the Stream API or GET API.

STREAM: Creates a single connection that receives data as it is pushed from twitter, this allows us to process the data in real time and not make continuous calls to twitter.

GET: Consists of standard HTTPS GET calls as and when we want to get information from twitter.

Whilst streaming gives us the most up to date data from twitter, it comes with additional complexity of having an open connection between your server and twitter that must be secured and kept alive which will generally require a bit more thought than an evenings coding. GET on the other hand suits my purpose well and allows me to call and handle data as and when I want.

For fullness I will show how to implement both:
----------------------------------------------------------------------------------------------------------
STREAM User Timeline:
client.stream('user', {track: 'GallagherAiden'}, function(stream) {
      stream.on('data', function(tweet) {
        console.log("got the following twitter message: " + JSON.stringify(tweet));
        var createdTime = tweet.created_at;
        createdTime = createdTime.substring(0,16);
        var tweetStream = createdTime + " " + tweet.user.name + " : " + tweet.text;
      });
      stream.on('error', function(error) {
        if(error){
          log.dashErr(error);
        }
         console.err("Could not get any twitter feeds");
        });
      });
    });
Here we use the twitter npm ".stream" functionality with the client defining that we want a "user"s stream and that that users handle is "GallagherAiden".

We then use "stream.on" to handle the incoming data which I then stringify and make suitable for my needs. i.e. in the following format:
Tue Sep 26 19:04 Gary Lineker - Gorgeous finish from @GarethBale11 gives Real Madrid the lead in Dortmund.
----------------------------------------------------------------------------------------------------------
GET User Timeline:
client.get('statuses/home_timeline', function(error, tweets, response) {
      if(error){
       console.log(error);
      }
var thisTweet = tweets[0];
var createdTime = thisTweet.created_at;
createdTime = createdTime.substring(0,16);
var tweetGet = createdTime + " " + thisTweet.user.name + " - " + thisTweet.text;
console.log(tweetGet);

Here we use 'statuses/home_timeline' which responds either with the tweets, an error or the response. This relates directly to the home timeline of the user who is linked to the application. There is no way to alter this.
The above code would give the same output as the stream in terms of formatting.
----------------------------------------------------------------------------------------------------------

The output of a single tweet is extremely large! 
{
    "created_at": "Mon Sep 25 06:22:53 +0000 2017",
    "id": 912200715552124900,
    "id_str": "912200715552124928",
    "text": "Photographing the Peak District in autumn \u2013 in pictures https://t.co/mHSVxMpNt2",
    "truncated": false,
    "entities": {
      "hashtags": [],
      "symbols": [],
      "user_mentions": [],
      "urls": [
        {
          "url": "https://t.co/mHSVxMpNt2",
          "expanded_url": "https://trib.al/hCIqP6P",
          "display_url": "trib.al/hCIqP6P",
          "indices": [
            56,
            79
          ]
        }
      ]
    },
    "source": "<a href=\"http://www.socialflow.com\" rel=\"nofollow\">SocialFlow</a>",
    "in_reply_to_status_id": null,
    "in_reply_to_status_id_str": null,
    "in_reply_to_user_id": null,
    "in_reply_to_user_id_str": null,
    "in_reply_to_screen_name": null,
    "user": {
      "id": 87818409,
      "id_str": "87818409",
      "name": "The Guardian",
      "screen_name": "guardian",
      "location": "London",
      "description": "The need for independent journalism has never been greater. Become a Guardian supporter: https://t.co/EWg9aqA7PQ",
      "url": "https://t.co/c53pnmnuIT",
      "entities": {
        "url": {
          "urls": [
            {
              "url": "https://t.co/c53pnmnuIT",
              "expanded_url": "https://www.theguardian.com",
              "display_url": "theguardian.com",
              "indices": [
                0,
                23
              ]
            }
          ]
        },
        "description": {
          "urls": [
            {
              "url": "https://t.co/EWg9aqA7PQ",
              "expanded_url": "http://gu.com/supporter/twitter",
              "display_url": "gu.com/supporter/twit\u2026",
              "indices": [
                89,
                112
              ]
            }
          ]
        }
      },
      "protected": false,
      "followers_count": 6791634,
      "friends_count": 1113,
      "listed_count": 54290,
      "created_at": "Thu Nov 05 23:49:19 +0000 2009",
      "favourites_count": 152,
      "utc_offset": 3600,
      "time_zone": "London",
      "geo_enabled": false,
      "verified": true,
      "statuses_count": 378671,
      "lang": "en",
      "contributors_enabled": false,
      "is_translator": false,
      "is_translation_enabled": true,
      "profile_background_color": "FFFFFF",
      "profile_background_image_url": "http://pbs.twimg.com/profile_background_images/704160749/ff996aa3bc2009a2f9b97cdd43e8b5b7.png",
      "profile_background_image_url_https": "https://pbs.twimg.com/profile_background_images/704160749/ff996aa3bc2009a2f9b97cdd43e8b5b7.png",
      "profile_background_tile": false,
      "profile_image_url": "http://pbs.twimg.com/profile_images/877153924637175809/deHwf3Qu_normal.jpg",
      "profile_image_url_https": "https://pbs.twimg.com/profile_images/877153924637175809/deHwf3Qu_normal.jpg",
      "profile_banner_url": "https://pbs.twimg.com/profile_banners/87818409/1491899430",
      "profile_link_color": "005789",
      "profile_sidebar_border_color": "FFFFFF",
      "profile_sidebar_fill_color": "CAE3F3",
      "profile_text_color": "333333",
      "profile_use_background_image": false,
      "has_extended_profile": false,
      "default_profile": false,
      "default_profile_image": false,
      "following": true,
      "follow_request_sent": false,
      "notifications": false,
      "translator_type": "regular"
    },
    "geo": null,
    "coordinates": null,
    "place": null,
    "contributors": null,
    "is_quote_status": false,
    "retweet_count": 1,
    "favorite_count": 3,
    "favorited": false,
    "retweeted": false,
    "possibly_sensitive": false,
    "possibly_sensitive_appealable": false,
    "lang": "en"
  }
 ----------------------------------------------------------------------------------------------------------
I have implemented this in a way that calls from the server using a "setInterval" ever 60 seconds which keeps me within my rate limit:
setInterval(function(){
  getTwitterFeed();
}, 60000);
I then do a web client GET of this data in order to populate my dashboard. I'm currently working with the last five tweets and will probably post how I'm doing that in a later post.

For now. Adieu. I hope you enjoyed reading, feel free to tell me where I'm wrong and how I can improve. you can email me at aiden.g@live.co.uk  or leave me a comment.

Twitter Feed on my dashboard, reloading every sixty seconds

Thursday, 17 August 2017

Selenium Tips - Node JS

So I've recently started making a few web user interface tests that can't be manually achieved with API calls. I was looking forward to practising my Node JS so thought I'd make it using that framework.

Here's a few little functions and commands to both get you going or help me remember what I've done:

Firstly, Selenium is a web browser automation software that can be run on the browser, in an application such as node or as an IDE. find out more HERE

Secondly, Node JS my desired programming service of the day is an open source platform built on a javascript runtime. There's a great youtube introductory video HERE

So... Getting Started. So I had installed node and created a dedicated directory.
I navigated to my new directory and installed selenium-webdriver:
 npm install selenium-webdriver
I then created an application:
echo "" > myNewApp.js
In the application I referenced selenium-webdriver and then used selenium builder to create a new browser driver like so:
const selenium = require('selenium-webdriver'),
    By = selenium.By,
    until = selenium.until;

var driver = new selenium.Builder()
    .forBrowser('chrome')
    .build();

var driver = new selenium.Builder()
    .forBrowser(‘firefox’)
    .build();
You'll either want different variable names for your different browsers or to only call one at a time!
From here on there are a whole bunch of commands we can run using driver.Command you can see a  list of some of the commands HERE.

The next thing to do is to perform a GET function on the webpage you want to navigate to in your automated test.
driver.get('www.myUrl.com').then(function(){
   console.log('Navigating to my page');

});
From this page we can perform a whole bunch of commands:
Populate an object:
driver.findElement(By.name(‘red’)).sendKeys(‘ABC’);
Clear a populated object:
driver.findElement(By.name(‘red’)).clear();
Make a hidden object visible:
driver.executeScript("document.getElementsByName(‘red’)[0].setAttribute('type', 'text');");
Using the drivers .then function will let you wait for something to occur and then perform the next action, such as wait for 2 seconds:
.then(function(){
  driver.sleep(2000);
});
You might want to click a button, all you need to know is the button class name. One way of doing this is to "inspect" the webpage from the browser.
Click:
driver.findElement(By.className('btn btn-lg')).click(); 
To quit the browser:
driver.quit(); 
The one thing I wasn't able to do was make a webpage stop loading to allow me to process a page that didn't stop. Let me know if you've done this before! I'd love to hear how you did it. 

Friday, 27 January 2017

HTML Image Arrays

I recently started to play with php and building a website. What I really wanted to do was to create a checkbox list of a certain category. Place names and the like. But it was a lot of repetitive typing, so I toyed with writing an array. 

The array was fairly simple to create but didn't look very nice. So I instead decided that images would better represent whatever it was I was displaying.
My first play (getting bits and bobs off the internet) looked something like this;

Array of images loaded side by side
<html>
<body onload="buildImage('slct1');">
<script>
   function buildImage(slct1) {
       
        var images = ["a.jpg","b.jpg","c.jpg","d.jpg","e.jpg"];
                 var s2 = document.getElementById(slct1);
                 s2.innerHTML = "";
                 var optionArray = ["a.jpg","b.jpg","c.jpg","d.jpg","e.jpg"];
                 for (var option in optionArray) {
                     if (optionArray.hasOwnProperty(option)) {
                         var pair = optionArray[option];
                         var img = document.createElement("img");
                         img.src = optionArray[option];
                         img.height = 410;
                         img.width = 327;
                         s2.appendChild(img);
                        
                         var label = document.createElement('label')
                         label.htmlFor = pair;
                         label.appendChild(document.createTextNode(pair));
                     }
                 }
      }
</script>
</body>
</html>
********************
Lesson 1: Make sure you're using " instead of 
      ********************
For the sake of playing. I was then able to put them vertical by adding the simple line:
            s2.appendChild(document.createElement("br"));
on the last line of the if statement within the function.
                         ...
                         var label = document.createElement('label')
                         label.htmlFor = pair;
                         label.appendChild(document.createTextNode(pair));
                         s2.appendChild(document.createElement("br"));
                     }...
Array of images loaded vertically
********************
Lesson 2: There is no need to over complicate what can be simple.
      ********************
All is going well. Though I'd prefer to allow the user to cycle through the images. Again using an array so as to keep reduce the amount of manual labour I need to do to expand the website.

The next step is to load the first image in a single space and then add buttons to allow you to move from the previous image to the next image.


The above images show me cycling through the image letters until we get to "F" which isn't in the images array so the unloaded image icon appears where an image should be.

<html>
<body onload="buildImage();">
<div class="contents" id="content" align="center">
<button onclick="previousImage()">Previous Image</button><button onclick="nextImage()">Next Image</button><br>
</div>
<script>
    var images = ["a.jpg","b.jpg","c.jpg","d.jpg","e.jpg"];
    var index = 0;

    function buildImage() {
      var img = document.createElement('img')
      img.src = images[index];
      img.height = 410;
      img.width = 327;
      document.getElementById('content').appendChild(img);
    }
   function nextImage(){
      var img = document.getElementById('content').getElementsByTagName('img')[0]
      index++;
      img.src = images[index];
      img.height = 410;
      img.width = 327;
}
    function previousImage(){
      var img = document.getElementById('content').getElementsByTagName('img')[0]
      index--;
      img.src = images[index];
      img.height = 410;
      img.width = 327;
    }
</script>
</body>
</html>

This is solved by adding the following code: index = index % images.length;
This loops the images forever and ever and ever and ever and ever and ever.
********************
Lesson 3: Don't test this functionality with 100 images.
      ********************
A final thought.. you can implement other functions to be attached "on click" to these images as they load. In the image properties just add: img.onclick = function(){FUNCTION(param1, param2)};

So happy coding everyone and many thanks to the following:
http://www.w3schools.com
http://stackoverflow.com/questions/13330202/how-to-create-list-of-checkboxes-dynamically-with-javascript