Building IBM ACE Bars using Azure DevOps and a Windows Build Server

For the last few months I have been looking at the most appropriate way to build ACE artefacts. This ranges from what makes good practice for ensuring consistency, what allows the best governance procedures to be followed and what is the best way to isolate the build and deploy of the artefacts?

One of the more common pipeline tools that is being used is Azure DevOps and so I've been playing with the mqsicreatebar, mqsiapplybaroverrides and mqsideploy commands in IBM ACEv11. The following are steps I wanted to achieve to get deployable units in ACE:

• Build a Base BAR file
• Apply environment specific overrides to the Base BAR to create an environment BAR file

To complete the build process we need to be able to run ACE commands. To do this we could use an external Server but when using a Windows machine there are certain ways these commands might be run. 

The following are some tips on how I completed the steps using Azure DevOps and Windows in my demo building scenario.

Method of Execution

Having decided to produce a script to be run on the Windows machine the first decision was whether to run as a batch script, bash script or PowerShell. 

• PowerShell is the Azure Cloud preferred method and comes with the PowerShell pipeline
• Batch files are out of the box, well understood by most and so likely to be easily supported
• Bash scripts are cross platform, easily portable and possibly better understood than batch scripts

In the end I chose bash because of the portability and ability to move across Cloud and Operating Systems with more ease and increases the number of people who can edit and support the scripts in future.

Running ACE Commands on Bash on Windows

Bash is not currently native to Windows machines and as such you have to run a bash environment on the Windows system. This will usually be Git Bash so to run in the Build pipeline in Azure using the command line task:

“C:\Program files\Git\bin\bash.exe” myscript.sh

 In order to run ACE commands, the ACE profile needs to be setup using the mqsiprofile command. In the Build pipeline in Azure using the command line task, run the following:

“Drive:\InstallPath\server\bin\mqsiprofile” && “C:\Program files\Git\bin\bash.exe” myscript.sh

Check the format of each ACE command 

It's important to check the format of each of the commands in ACE as some of the command are already executable in shell e.g. cmd format and do not need the file path specificying in the bash script, others are in BAT format so need the full path specifying. One example of this and specific to our needs is the mqsiapplybaroverrides command which needs the .bat extension name when being called in the bash script and also needs the full path to the file.

Passing Azure Variables to the Script

Azure variables need to be passed to the script via the command line task. Azure uses the following format to reference existing variables $(myVariable). Variables can be passed in using the Variables tab e.g. appName (to specify which to build) or predefined Azure variables e.g. System.ArtifactsDirectory

Example buildACE.sh $(appName) $(System.ArtifactsDirectory)

#!/bin/bash

appName=$1

buildAddress=$2

mqsicreatebar -data ${appName} -b ${buildAddress}/${appName}_Base.bar -a ${appName} -deployAsSource

drive:/installPath/server/bin/mqsiapplybaroverrides.bat -b ${buildAddress}/${appName}_Base.bar -p ${appName}/dev-overrides.properties -o ${buildAddress}/${appName}_DEV.bar -r

The following example will take the application being built and the build output location (Artifact) as parameters which are used in the bash script. 

As you can see the mqsicreatebar can be ran without specifying a path because we have the mqsiprofile set, but the mqsiapplybaroverrides needs thee full path to the batch file defining.

The output from the above script would be a base bar and a dev bar for the application name supplied in the Azure Build. Further scripting can be implemented to produce BARs for more environments.

IBM MQ – AMQ3817E – DRBD Errors

 Recently when rebuilding an MQ Estate, the previous deployment was torn down and a new installation and configuration was deployed to the same physical Virtual Machines. However, when it came to creating the new Queue Manager there was an error thrown.

 

The reason for the rebuild was to upgrade the version to IBM MQv9.2 which includes some changes to the way RDQM pacemaker and drbd kernel modules are installed.

 

Command:

crtmqm -rr p -rt s -rl thisIP -ri otherIP -rn otherHostname -rp port -lla -fs 20 -lp 6 -ls 3 qmname

 

Error:

AMQ3817E: Replicated data subsystem call '/usr/sbin/drbdadm new-resourceqmname 0 --auto-promote=no' failed with return code '20'.Command 'drbdsetup new-resource qmname 0 --auto-promote=no' did not terminate within 5 seconds

AMQ3812E: Failed to create replicated data queue manager configuration.

 

To debug I did the following:

1.     Checked the MQ Error logs

2.     Check there were no outstanding processes that could be blocking the crtmqm command

3.     Check the correct drbd kernel module was installed on the server. (modinfo drbd)

 

There were no errors in the MQ Logs, no processes running for drbd and the correct kernel module was installed. Sufficiently stumped I ran lsmod drbd | grep drbd on the server and compared it to another environment.

 

There was no drbd_transport_tcp module on the output of the environment with the issues only the drbd module. When there are no RDQM Queue Managers running lsmod drbd | grep drbd hould return empty, which seems to suggest the drbd module was hanging. 

 

To resolve this there are two possible actions:

a.     Reboot the machine

b.     Remove the remaining ‘hanging’ drbd module using rmmod drbd

 

After completion, run lsmod drbd | grep drbd and the result should be empty. Try to build your Queue Manager again and it should create successfully.

Untangling Governance Definitions - Patterns, Blueprints, Templates, Standards and Reference Architecture

By Aiden Gallagher & Peter Reeves

*Disclaimer: These are our own definitions and might not map to your specific organisation. And, of course, these are our own opinions, and not necessarily those of our employer

Introduction

For the last few weeks, Peter and I have been talking about Integration Patterns, blueprints and architectures to try and boil down a common definition which we can use when talking about these concepts.

We found that halfway through a conversation we would start merging elements and classifying the same integration item as different things.

To remedy this, we decided to write down the definitions as a reference point that we could refer back to. In this blog post you will be able to see these definitions, and when we talk about Patterns, Blueprints, Templates, Standards and Reference Architectures you will know what we mean too.

Why do we care?

Before we share the definitions, it’s important to know why we care about the words themselves. We want to be sure that when we are discussing concepts that we are all referring to the same things and in the same way, related to how artefacts fit into the wider organizational picture and the correct governance procedures when designing systems.

Definitions

Below you will find our definitions and with them an example of how these are applied outside of the computing world. In this example, we will apply them to a housing project and “dwellings”.

Pattern

Definition – A pattern is a refined repeatable concept which is a solution to a common problem.

Example – Given the common problem of a human needing a place to live; have a roof over our heads, running water, sanitation, heat and the ability to cook, a pattern standardises the way we build a ‘dwelling’ that acts as a solution to these problems.

Blueprint

Definition – A blueprint is a detailed design for how to implement a pattern using a certain technology.

Example - A blueprint of the dwelling pattern could be a terraced house which provides dwellings against a limited land space. It could also be a bungalow which provides accessible dwellings for those who cannot use an upstairs.

Templates

Definition – A template is an implementation of a blueprint with further details on the specifications for items of that type. It might be followed exactly, or it might be altered to fit the need of the circumstance.

Example – A terraced house in Darlington was previously built with certain specification. The designer wants to provide more garden space as such they can tweak the template of a terraced house during design. But the template largely remains the same.

Standards

Definition – Standards are an approach, design or method that is applied across all interconnected systems and applications to ensure consistency of approach, and this benefits support, testing, and development.

Example – The plug sockets in all houses are the same size, as that meets the requirements for the majority of household appliances. This means all devices are interchangeable, which in turn means less customisation during build and hence is cheaper to build and more convenient for the end users.

Reference Architectures

Definition – A reference Architecture is a set of documents or information that describe a topology or system that can be used for common and related patterns. The information, if followed, will provide commonality across deployed templates and blueprints.

Example - A large plot of land is bought for building development requiring sewage, electricity, water, gas, and road network. A reference architecture describes how this infrastructure can be setup to accommodate different home styles (patterns) the instructions for building each home (blueprint), and potentially any customisations to the blueprint to fit other requirements (templates, and templated usage). Throughout the whole build process the same style of roofs, doors and windows are purchased (standards).

Integration Example

In the integration world we might apply the above definitions to the following use case.

A hybrid-cloud deployment strategy has been documented which shows how availability, disaster recovery, networking, and deployments of servers, and various business functions might be achieved in a reference architecture for that system.

The cloud deployment has a number of topologies (or building blocks for topologies). These are used as necessary to build integration patterns (e.g. request-reply which needs to solve some high-level problems like: access to a database, transactionality, reliability etc.)

Having selected the reference architecture which will support specific patterns, a blueprints to implement desired patterns using those cloud offerings can be written by the organisation – this could form a high-level design of how the new system will look like, this is further refined and customised in the low-level design. 

Looking at a Hybrid Cloud implementation, there are some orchestrated deployments packaged as part of a cloud offerings to solve specific business needs, which is in itself like an off-the-shelf blueprint combined.

To build the system, a series of templates are used for each cloud product with necessary minor tweaks that meet small nuances of the pattern and blueprint requirements. Throughout all these processes the organization’s standards are used to ensure naming conventions are the same, disaster recovery can be performed by any of the support teams, that port mappings follow the same style for easier communication with the network teams, and any other core business procedures are followed.

The end result is a system that was built quickly, in a reusable, extensible, and familiar way and that is now supportable by the wider organization.

Benefits

By using Patterns, Blueprints, Templates, Standards and Reference Architectures there are obvious tangible benefits:

Standardisation

·      Less complexity when implementing solutions as everything is known.

·      Cost reduction of multiple teams solving the same problems repeatedly (why reinvent the wheel?)

Reusability

·      Once the patterns, blueprints etc. have been shaped and refined, it becomes easier to reuse them than to start the building process from scratch- saving time and energy exponentially.

·      They are easy to use.

·      They are easy to deploy.

·      They are easy to support.

Familiarity

·      There is less project-specific knowledge required when people switch roles as the core patterns (and blueprints) are the same.

·      Individuals are able to get the 'gist' of what is happening in a new project- even for customised docs, build etc- because everything looks and feels the same and was built with the same principles, even if the low-level implementation is slightly different.

·      This familiarity makes it easier and quicker to fix, debug and test new projects.

·      This leads to a reduction in dependencies on individual resources.

Future Proofing for the whole Organisation

·      Improving the pattern by one team improves for all.

·      The whole organisation is carried into modern practices together because of the usage of patterns, blueprints etc.

Limitations

There are some limitations to the usage of Patterns, Blueprints, Templates, Standards and Reference Architectures which tie closely together:

Complexity

·      Sometimes complexity is required, maybe because something has never been done before, or there are new nuances such as security regulations that make a solution difficult to implement using existing patterns.

·      Trying to fit the patterns into these use cases might cause more harm than good e.g. over-engineering, barrier to release time, tendency towards “You aren’t going to need it” (YAGNI) behaviours and a bloated system.

Blocker to change

·      It can be hard to change a pattern because it’s a known entity which people are familiar with. The pattern then becomes a tool to block improvements.

·      It can be hard to get change because the benefits of a new system might not be understood by the wider organisation who must accept these changes.

·      Patterns become tied to governance process. This can undermine the governance process which can be deemed as ‘red tape’.

Stifling Innovation

·      Working only to the pattern can stifle the creativity which comes naturally to a lot of people in an organisation. This means that the pattern and its prescriptive, restrictive methods can lead to a lack of innovation and improvements.

·      It can be hard to get investment for new ideas because there are no any proven benefits, which is perpetuated by a lack of ability to improve existing patterns.

For all these limitations, it is possible with good organisation practices to be able to work around these requirements. This might be by encouraging minimum viable products (MVPs), allowing colleagues to use new tools in their day to day life, and having governance process that look to solve organisational problems and ongoing reviews of current business processes.

Conclusion

In this blog post, we have given our definition of Patterns, Blueprints, Templates, Standards and Reference Architectures, along with a real-world example and a computing example alongside an exploration of the benefits and limitations of using them.

Have any thoughts? Do you vehemently disagree with our definitions? Have a better computing example?  Get in touch to tell us what you think

Working from Home – Aiden’s Top 10 Tips

About a year ago I began to work from home fairly regularly, spending at least 1 or 2 days at home. Since October, I had been working from home 3 days a week and commuting to the office the other two days. Since Coronavirus struck, like near everyone else in the country, I’ve been working from home full time.

The following article looks at how I have transitioned to working from home and my top 10 tips for staying sane and productive whilst maintaining a work life balance.

1.   “Commute” to work

We first tried this in October when we moved home. We found a local walking route of about 10 – 15 minutes long. The morning walk helps get your brain into the work mindset, start thinking about what you need to do that day and gives a change of scenery instead of being stuck in the same room(s) all day every day.

An added benefit is that it also means you will already have accomplished top tip 2.

2.   Get dressed for work (in something comfortable)

I’ve sat in my pyjamas for work many a times and by about midday I always start to feel lethargic and unmotivated. Getting up and getting dressed helps to switch to ‘work’ mode. There’s no need to wear a suit but wear something appropriate – to help mentally separate work from not working and for any video calls you might have. But, make sure you are comfortable.

3.   If you don’t have an office – ‘prepare your workspace’

Not everyone has an office at home – I don’t either. For me, I work at the kitchen table. However, when I am working at the table, it doesn’t feel like I’m at ‘home’ and when I’m eating at the table it doesn’t feel like I’m at ‘work’.

For example, for me, to separate the two, I transform the kitchen table into ‘work’. I move the table mats and the vase that normally rests there and swap it for my laptop, whiteboard and tea. This again allows me to switch to ‘work’ mode.

I typically try to avoid the sofa as a workspace, in a previous house I ended up turning it on a couple of times and have sucked hours out of my day from productivity which I had to make up later. There’s time for TV after work. If you can though, I would 100% recommend buying a radio as it helps to recreate that background chatter of the office and has the additional benefit of reducing the feeling of loneliness that can build when working from home alone.

4.   Be consistent with lunch

This one is nice and simple. Try and keep a routine; same start time, same lunch time, same finish time. The primary one is to be consistent with lunch time. It helps keep the brain ticking over, gives you a deadline to work towards and gives a good structure to the day. 

Just like at work, I would say that lunch shouldn’t be eaten at the desk, if you do, it makes you think you should carry on working – even if you have good intentions to take a break. Go sit on the sofa, or if the weather is nice, out in the garden/balcony which gives the brain a welcome break from work.

5.   Clock off at your pre-defined time

Its key to ensure you finish at the time you have set yourself. If you work 8 hours a day and start at 9, make sure you finish at 5. I’ve always found an implicit guilt that I haven’t done enough work throughout the day and need to work a little bit more. Even though I know it’s not true (I’m actually more productive at home) I always hang on a few extra minutes. 

This doesn’t give the brain enough time to rest from day to day or allow you to think about the work tasks that needs your brain whirring – which most will do in their down time. So, shut off the laptop sharpish when work finishes. 

6.   Happy scenery setting

I’m a firm believer that there are four key things that make a workspace a good place to work. 

1.     A Plant

2.     Natural Light – preferably so you can see out of a window

3.     Desk items / wall decorations – somewhere to jot notes, a picture or even a calendar

4.     Access to good tea/coffee

7.   Communication is key

Where the home office differs mainly from a collective office is the ability to talk to your colleagues – and other humans in general. There are a lot of interactions you miss at home; the hubbub of the office, picking up on nuggets of knowledge from background noise, project specific conversations and the water cooler conversations about home and personal life.

As I mentioned before, a radio can help recreate the ‘hubbub’ or office noise, but it’s important to setup conversation channels on your conversation application of choice (e.g. slack) for different purposes and to encourage input to them. These might be:

·      A project channel – say what you’re doing on the project, why you are doing it, what help you might need or any nuggets of information you have discovered

·      General chat – this might be with for your project team, organisation team or even the whole company just to discuss the geographical news, big stories of the day, funny articles or something to inspire

·      Can the slack chat be a meeting? – it’s worth asking if you can make a couple of your one on one conversations a phone call just to get that human verbal interaction. Tone and emotion can be lost online so it’s good to talk over the phone.

8.   Take breaks

Short and simple – take regular breaks. 5 minutes between meetings, ten minutes to make a tea and call a colleague, 5 minutes after finishing a difficult task to unwind. Whatever you do, take regular breaks.

9.   Limbering up

For anyone who has watched Zombieland, the importance of staying limber is already obvious. For everyone else, getting up, stretching, moving about during the day helps reduce the risk of bodily niggles. It’s good for the body, and if it’s good for the body, it’s good for the mind.

A good one to do is to join calls on the phone if you don’t need to see the screen and walk around the house. Not much space? Doing stretches on the hour every hour can be done stood at your desk or even whilst the kettle is boiling.

10.        Task list

For my ‘office’ I have a little whiteboard. It has my long-term goals on it, my side projects (like articles or code), my day to day tasks and things I need to do for the house (although, that does mean that I’m reminded daily that I still haven’t rung a window cleaner). 

These keep my deliverables in focus, allow me to keep refocusing my priorities and means I can quickly answer the ‘what is on your plate now’ when the project manager calls me. This also means I can accept or reject new tasks based on how busy the whiteboard says I am.

This means you aren’t taking too much on, aren’t having to consistently work late and thus allows you to understand what needs to be done to plan your work and home tasks for life balance.

Conclusion

So, these are Aiden’s Top 10 Tips for working from home. I hope it gives you some good ideas for how to stay happy when working at home. Any tips of your own you want to share? Let me know