Category

Enterprise PowerBI

UiPath & Azure: Data Driven Efficiency

By | Automation Initiation, Data Platform, Enterprise PowerBI | No Comments

At Talos our goal is simple: to deliver data-driven efficiency. In our experience, the best way to achieve this technically has been through the combination of UiPath & Azure – two very powerful technology stacks that when paired properly will deliver immense value. This blog post examines the relationship between UiPath & Azure, a typical design & use-case, and the data driven benefits of leveraging both together.

     

 

UiPath & Azure

UiPath is the world’s leading RPA company delivering a suite of automation technologies targeted at the enterprise level. Azure is Microsoft’s cloud platform, offering a vast array of services to customers to meet their complex needs. Given the technical strengths of each, it is possible to craft a solution that utilizes both to achieve optimum benefits, and in our case, deliver our goal of data-driven efficiency.

In numerous projects we have worked on, there has been a typical requirement to deliver reporting based on the combination of data from a variety of sources. This simple use-case can actually be quite complex, especially when dealing with legacy data source systems, large volumes of data and a significant reporting consumer-base. However, these requirements can be met with a very straight-forward UiPath & Azure solution design:

UiPath & Azure

This design is composed of the following individual elements:

  1. Azure VM’s
    1a. UiPath Studio
    1b. UiPath Unattended Bot
  2. Azure Blob Storage
  3. Azure Data Factory
  4. Azure DevOps
  5. Azure SQL Database
  6. Power BI

Each of these elements have a specific role in the solution, and fit in easily with each other, making this solution extremely robust, easy to initiate and  scalable.

  1. Azure VM’s

Azure Virtual Machines (VM) are the image service instances that provide computing resources that behave exactly like physical infrastructure, except virtually. The benefits of Azure VM are the on-demand nature of the product and the lower costs associated with management. In our solution, 2 of these VM’s are commissioned to provide separate environments for the UiPath tool:

1a. UiPath Studio is installed on the first VM (this is used as the dev/test environment). UiPath Studio is the development tool used to create and test automations. In our case, we use UiPath Studio to develop the automation steps required to retrieve data from the different legacy systems.

1b. UiPath Unattended Bot is installed on the second VM (this is used as the production environment). Once the automations are developed and tested, they are deployed to be executed by the Unattended Bot on this VM. By having a dedicated VM, this bot acts like a 24/7 worker and is available on the VM to perform any tasks it is instructed to do.

  1. Azure Blob Storage

Azure Blob storage is the scalable and secure object storage service used to store data in a variety of formats. Blob storage is very robust and can scale very easily to suit storage needs. In our case, we leverage Blob storage as a staging area for the bot to store the data it has retrieved, and also serves as a source for the Azure Data Factory pipeline later on. UiPath can communicate with Azure Blob storage natively, making this integration easy and reliable.

  1. Azure Data Factory

Azure Data Factory (ADF) is the data integration service built for complex ETL projects. The benefits of ADF are the ease-of-development as well as the powerful capabilities that allow it to handle complicated data transformations in large volumes. In our case, we use ADF to create a pipeline to move data from Blob Storage into the Azure SQL database, whilst performing sophisticated transformations. Although UiPath possesses some ETL functionality, they lack the advanced capabilities of ADF regarding transformation and speed. For this reason, ADF is a must have in our solution.

  1. Azure DevOps

Azure DevOps (ADO) is the collaboration tool for software development offering work tracking, source control and continuous integration/delivery. ADO allows projects to be managed effectively and collaboratively. In our case, ADO acts as the project management tool and code repository for the ADF pipelines. Although UiPath has a native integration with ADO, it also possesses the source control and deployment capabilities out-of-the-box, and therefore can be managed within the tool itself.

  1. Azure SQL Database

Azure SQL Database (DB) is the cloud-based database service built on the SQL Server engine. It has a wide range of deployment options, making it very easy and effective to manage. In our case, this is where the transformed data is stored and made available for analysis. The scalability, availability and deployment ease of Azure SQL DB make this very attractive for enterprise-level analytics and reporting.

  1. Power BI

Finally, whilst not an Azure service itself, Power BI is used as the reporting tool to develop custom reporting on the data in the Azure SQL DB. Power BI is an excellent reporting platform for enterprise-level reporting due to its scalability and self-service capabilities. With this tool in place, the data is modelled and reported to the business.

Although every project is different, the above represents a good design for a typical solution and is used by us as a list of ‘minimums’ that we need. For our work, this design is very common, but is also great as a default because it can be restructured very easily to suit any additional needs (for example adding Azure Machine Learning can be integrated into the above design as part of a CASSIE deployment). With the above solution, a business can very quickly leverage the benefits of UiPath & Azure working in unison and delivering data-driven efficiencies: businesses are able to get the insights and reporting they need without having to expend any staff time to deliver it.

If you want to know more about UiPath & Azure together, please contact us.

PowerBI governance

The need for good PowerBI Governance

By | Enterprise PowerBI | No Comments
What do we need PowerBI Governance for? Well, PowerBI has come a long way in the past decade – and yes, it has been a decade. It was released way back in July 2011! What was a humble reporting tool is now a complex ecosystem of self service analytics, data management and artificial intelligence. While the advancements in themselves are great, with greater complexity comes a greater need to understand and manage it through PowerBI Governance, especially at the Enterprise scale.

What needs governing?

In my experience across a range of implementations there are 4 key areas that need formal management:

  • Data
  • Security
  • Platform
  • Users

I’ll run through each of these below in more detail.

PowerBI Governance – Data

PowerBI governance data

Depending on the size and maturity of the organisation, data governance may already be in place. If so, this part of the design of the PowerBI governance model is simple. The existing policy can directly inform your approach.

However if it is absent, then part of designing the governance model includes creating it. In the early part of a deployment lifecycle, where PowerBI reports are simple and departmental, accountability for data is often pretty simple; the department that creates the report knows data owners and can manage issues as they arise. However once the reach of PowerBI scales these links become less clear.

If you are managing PowerBI centrally in an organisation, and a data item is suspect, how does it the investigation process work? How does the organisation decide on where to allocates its limited development resources if demands from different areas are in contention? How do we prevent disparate teams duplicating efforts? Who is accountable for maintaining the data catalog that makes assets discoverable? Who decides who can see what data when it is discovered?

All these questions fall under Data Governance and answering them is critical to ensure a well managed self service analytics environment.

PowerBI Governance – Security

Security

It is becoming cliché to say it, but data is a valuable asset. I’ve written before about the security risk associated with PowerBI desktop but like all data stores it has many attack surfaces and holes to leak from. Managing them correctly and actively is essential to prevent embarrassing and potentially expensive data leaks. As with all issues to do with governing technology risks, there are a mix of hard (technical) and soft (policy) tools you can bring to bear to the problem.

On the “hard” front are the platform level controls you can put in place. Some of these sit at a purely PowerBI level, such as the ability to limit insecure external sharing via publish to web. Other controls need the engagement of Azure administrators through deploying tools such as Azure information protection to keep an eye or lock on sensitive data.

On the “soft” front, giving education on the risks of data loss, costs of data leakage and how it happens is effective. It is not reasonable to expect untrained users to foresee the unexpected consequences of downloading and emailing a copy of a report. Most data breaches are caused by human error rather than any malicious activity. Education is key to this issue.

Thus the PowerBI platform needs to have basic guidelines established for its use, and then communicated around the organisation. It’s not a set and forget activity either – the technical environment and the business it serves changes. Once again, good ongoing governance is essential to keeping your platform and data secure.

PowerBI Governance – Platform

Platform

PowerBI is a complex beast. The admin portal overview page alone is now a half hour read! There are controls over the use of capacity, what is able to be shared and how, what visuals can be used and even how the portal can be customised to match your organisational branding. Then there are the requirements to monitor the platform for usage and capacity. This includes on-premise data gateways – which aren’t part of the services management capability.

On top of this is decisions to make around the usage and administration of Workspaces. End users should be given flexibility to self serve, but designing controls around data access is vital. Preventing scenarios like someone accidentally sharing HR reports with everyone’s salary details in them requires planning.

While it is IT’s role to administrate the platform, the data on it is a business asset and so the governance process needs to be a joint effort.

PowerBI Governance – Users

Users

Last – but definitely not least – are the users of all this capability. Rolling out PowerBI across the enterprise means you need to support and enable the users through a managed program. How are you establishing your Centre of Excellence to ensure quality content gets built and delivered to stakeholders? How can you drive an internal PowerBI community to support the growing capability of self sufficient analysts? Who will design and manage the training program to the different audience types that need to be catered to?

Plus of course is the balance of how to licence everyone. Is PowerBI premium the right option, or can you get by on PowerBI Pro licences for now? What is the trigger to make the change?

Driving the platform to deliver value to the business is not something that will happen by accident. Data literacy is not a native skill to many workers. The process of infusing that in your business needs to be managed.

Summary

This article aims to highlight some of the key concerns in governing PowerBI across the four dimensions of Data, Platform, Security & Users. As you have undoubtedly been gathering there has been a growth in complexity of the ecosystem. This has led to a similar complexity in terms of how the platform and its use needs managing.

If you need to hit the fast forward button on all this, we can help. All this complexity falls under the first step in our PEBBLE Enterprise PowerBI methodology – Plan. Our PowerBI Governance framework addresses these concerns – and more. Please reach out if you would appreciate some help.

Popular BI – State of the Self Service BI market Jan 2022

By | Data & AI, Data Visualisation, Enterprise PowerBI | No Comments

Popular BI Tools in 2022

This is an update on a series of posts I have written over the last few years looking at the relative popular BI Tools (2019’s is here). I did try to expand my search to cover more platforms but there are still only three that seem to really matter.

Trying to get actual usage figures of any Self Service BI tool is pretty difficult – none of the big vendors will willingly release figures – and if they did they’d probably be suspect.

But in analytics sometimes there are useful proxies which, while not as accurate as hard numbers, can give a useful perspective. In this case, we will use our friend, google trends:

Graph of search for Popular BI tools

Tableau is Red, PowerBI is Yellow & Qlik is Blue.

A quick note on methodology – I’m looking for Qlik (Software Company), Tableau (Data Visualisation Company) and Power BI (Software). I restricted the search to the US because Tableau is also French for “Table” so a worldwide search gets noise from that as well.

State of the Self Service BI Market

What you can see is that PowerBI has caught up with and slowly started overtaking Tableau as of early 2021. Tableau no longer appears to a leader in the self service field, with PowerBI looking to dominate. Qlik is continuing its slide into irrelevance. From this data we get a bit more insight – search volume across all three has been declining from its peak in 2019, now at 80% of its height in 2022.

Next up if we look at market share rather than sheer numbers, we see something interesting. In 2016 Tableau held about 70% of the searches, which has now dropped in 2021 to 46% (and still falling). More interestingly Qlik’s share has fallen to about 5% of the market (compared to 18% in 2015) but Power BI’s share has grown to about 49% (and still rising), compared to 8% in 2015. From the chart there was quite a change in velocity in Tableau searches from early 2020 (in line with the acquisition by Salesforce). It is also worth underlining the fall in share for Qlik is also based on declining search volumes overall.

What does the future hold for Popular BI tools?

Based on trends to date I would make the following headline predictions for the Popular BI tools market in 2022:

  • The Self Service BI market is not as hot as it once was, the hype having moved on
  • Power BI will continue to experience strong growth and will consume more market share, whereas Tableau’s growth will fall
  • QlikView will continue to decline, and no major competitors will arise either

These findings confirm my view held since 2018 – confirmed by my market feedback – that Qlik is a dead platform walking. It’s failed to catch up with market changes and is simply a legacy product. It’s squeezed between the two with no compelling reason to be chosen. However with Salesforce acquiring Tableau, it seems to have gone into a vendor locked decline (in a similar manner to products that have fallen into the SAP or IBM fold in older days). PowerBI has an almost empty field in which to compete.

Competing with PowerBI is going to be a tough proposition as any challenger will need the all encompassing Data Platform juggernaut of Azure behind it (excuse my hyperbole) which gives it the enterprise capability needed to succeed long term. Effectively only Google or Amazon could hope to do so.

However, Google and Amazon’s offerings, (Data Studio & Quicksight respectively) still lack much market traction at this point. The acquisition of Looker seems not to have impacted Googles presence much, despite having 3 years to do so.

My previous predictions have pretty much held true, but as Yogi Berra famously said, “It’s tough to make predictions, especially about the future”. I can be certain that it will be another exciting year in Data & AI. We specialise in Enterprise PowerBI so contact us if you need some help on your journey.

3 faces of Enterprise PowerBI Training

The 3 faces of Enterprise PowerBI Training

By | Enterprise PowerBI | No Comments

Since Self Service BI became an actual thing we have advised many organisations on how to roll it out successfully and give the best ROI as part of our Enterprise PowerBI Training solutions. A key mantra for me has always been to tune the content to the audience. After all, you book The Wiggles for your children’s party, not a heavy metal band (well, unless they are into Hevisaurus).

Over the course of working with organisations to define these audiences, consistently there are 3 personas that come up, each with their own needs with regards to enablement in terms of data and training. These 3 personas are:

  • The Actor
  • The Creator
  • The Author

Lets have a quick walk through these roles!

The Actor

An Actor is reading from the script, interpreting what is in front of them but not changing it.

This user makes up the bulk of users in most organisations. Their use of data and reporting is as an input to drive or inform decision making. The individuals in these roles can range from the CEO who needs to have an at-a-glance dashboard of their organisations performance, to a field sales worker who needs to know the buying profile of their next client.

The level of interactivity with any reporting will be basic – maybe selecting a few options or filters, perhaps drilling down a pre-set path to get some finer detail. Consequently the degree of training and enablement they need is fairly light. Key information for them is where to find the report, what the data on the report actually means and what buttons to press.

The Creator

A Creator builds an vision from the script, taking it to design their performance.

This type of user is actually one of the most important in terms of organisational impact. These are the people that are tasked with generating content for the Actors, and as such have a deep understanding of the data in their domain. These are the people tasked to work with the technology experts to build out the data models that drive much of the self service capability.

These users really get into the guts of the data and understand it in depth. When an Actor asks for explanation on a particular data point they are the ones that have to investigate. The technical training they need focuses on content creation and publishing. The enablement needs to cover things like business analysis skills, Master Data Management and Data Cataloguing.

The Author

The Author of course writes the script, starting from very raw materials to build a story.

Most people calling themselves PowerBI experts sit here, and most organisations have a handful of them – they are not a big population (though in my world are a vocal one!). They sometimes fill the role of creator but more often than not are trying to make sense of the organisations data, how it interlinks and where the hidden treasure lies. They “self-serve” from the raw data, constructing new ways to get insight into the organisations performance.

Here enablement focuses on technical capability as they need to understand how to manipulate and interrelate data that may be in less than ideal forms and certainly hasn’t been through an enterprise cleaning process. Data Catalogues support them in the discovery process.

To wrap

The key message to take from this post is that when rolling out Enterprise PowerBI Training at scale these different communities and capabilities need to be addressed – and in different ways. There is no value in sending all of your team on advanced PowerBI training if the skills learned will never find a practical application. Similarly if you build it, they won’t come – you need to guide them there.

Good luck, and if you need some help or advice, please get in touch.

PowerBI Deployment – it grows like Mushrooms!

By | Enterprise PowerBI | No Comments

PowerBI Deployment strangely coincides with one of my other hobbies. I have had a lifelong interest in hunting mushrooms, and was recently gifted a copy of an Australian Field Guide, which helps a lot as my existing knowledge is UK based and not just irrelevant but potentially deadly.

Now, first the science bit: Mushrooms themselves are actually just the most visible part of the whole fungi, which is actually a huge network of mycelium growing in the ground, or in an old tree trunk or whatever its preferred food is. A mushroom is the just the fruiting body of the fungus itself. By the time you see a mushroom, the fungus itself is well established and growing healthily.

PowerBI Governance Mushrooms

PowerBI Governance Mushrooms in the wild!

PowerBI – by the time you see it, it’s already established

From an organisational perspective, PowerBI Deployment and mushrooms have a lot in common. By the time a leader becomes aware PowerBI is in the organisation by having spotted a wild dashboard – it usually means it’s got into many parts of the organisation in a wild and unmanaged state.

This isn’t a problem in itself; a few wild gems here and there are good thing to find and can add value to the business. However as with wild mushrooms, telling the good from the bad is not a simple task. Superficially one report can look very much like another. However one can contain the poison of bad data or incorrectly applied business rules. There’s no way to tell from just looking at it.

This why we farm our mushrooms

To ensure our mushrooms on toast doesn’t end up killing us, we farm mushrooms to make sure we get a safe, edible species to eat. Similarly for PowerBI, once we start consuming it we want to be able to confident of its provenance. This means that you need to move from a business led anarchic state to something more managed.

There’s a lot of models for delivering an Enterprise PowerBI platform but they boil down to 3 approaches:

  • Business Led, bottom up
  • IT Led, top down
  • Hybrid – IT Managed, Business Developed

Which one is right for your organisation will depend on your particular structure and culture. Typically the Hybrid model delivers best results though, as IT manages the technical components – building data sources and so forth – and Business manages the consumables – reports & analysis on that data. Each area plays to its strengths.

Applying governance to your PowerBI deployment

Managing a balance of responsibilities between IT and Business is never easy and so laying some ground rules is essential. Things you need to consider include:

  • Who is accountable for the validity of the data?
  • How to manage the service from a cost and capacity perspective?
  • What do you do to identify and manage security risks?
  • How will you enable adoption and effective usage?

All these things require careful thought and then enforcement through “hard” technical configuration and “soft” policies and guidance. There’s no right answer, and Microsoft’s continual updates mean the answer keeps changing. My experience with organisations from Local Councils to Global Law Firms to Resource companies has helped me know the right questions.

If you are struggling with an emerging wild mushroom PowerBI deployment, we’d love to hear from you and see what we can do to help bring them under control as part of applying our PEBBLE Enterprise PowerBI Methodology.

And a closing dad joke:

Q: Why does the mushroom always get invited to parties?

A: Because he’s a fun guy!*