Blog l AIMS Innovation

How to understand website performance in relation to your databases and middleware

Written by Ivar Sagemo | Sep 8, 2017 8:39:41 AM

At AIMS we focus on simplifying and automating business- and technical insight into complex and connected systems.  At the heart of these systems we most often find middleware platform / ESBs and databases hosted on physical or virtual servers on premise or in the cloud.  These systems have during the last 10+ years become the heart and brain of many critical business processes and historically primarily implemented by large organizations. 

But with the increase in software (and application integration) supporting the explosion in business processes driven by IT we increasingly see the need to understand other systems which are tightly connected to delivering the relevant business process.  Web servers originating transactions to back-end systems is a common scenario.  

While many of these systems are less complex than middleware and databases the same challenges apply as we have seen with middleware and databases.  Those challenges center on the difficulty of using traditional monitoring tools and human labor to identify problems in dynamic IT environments with "living" and cyclical behaviors.

How can you as a sys admin or IT ops understand if a certain behavior is an anomaly?  How can you track and analyse behavior of thousands of relevant parameters across several applications and see how an issue initiated by one component (and maybe on a trivial process) impact other (critical) applications sharing some (maybe unknown) resource?  That's where machine learning and AI plays a role.  

Why is Website Peformance important?

While being focused on the integration platforms and tightly connected systems we recognize the importance of web servers as a critical application to support origination of transactions.  These transactions originated from a web server could support a range of different business processes f.ex. supporting internal (productivity) solutions for a company or external revenue generating business processes exposed to customers.  Examples include e-commerce websites taking orders inc credit card payments, package/transport tracking,  retail loyalty card /coupon systems, airline booking systems or insurance claims.  Performance problems with these processes translate directly to lost revenues, lost customers, lost productivity, reputational damage and potential regulatory breach.  In others words - your ability to manage importantant business processes from a website through the back-end could have a material impact on the P&L of your company.

And that's why we developed support for Microsoft IIS - to tie web site originated requests with performance of middleware, databases, custom applications, other websites and perhaps something as simple as file count on a fileshare.

AIMS Agent for Microsoft IIS has been available for a few months and was developed based on specific request from customers who requested support for anomaly detection across BizTalk, SQL and IIS.  

How does the AIMS IIS agent work?

The AIMS IIS agent is as easy to install as the other AIMS agents and you should be all set within a few minutes.  More about the install is available on our support page here.  

If you are an existing AIMS customer you can reach out to AIMS Support and get help with a trial version of the IIS agent.  If you are not a current customer but interested in testing the IIS agent and other technologies you can request a trial.  You can also sign up for AIMS FREE which gives you free use of AIMS at certain limitations.

When you have the AIMS IIS agent installed you will see the Agent available in the Topology menu as shown below (Internet Information Services).

 

 

 

 

 

 

 

 

 

Further, your IIS will be available in the Topology overview as shown below.  The top left box in the 'boxed area is the IIS.

Now, what is the data we fetch from the IIS?  

For Application Pools we fetch : Total threads, Active requests and Requests

For Servers we fetch : Requests Queued

For Sites we fetch : Logons failed, Bytes received, GET requests, Logons succesful, 5xx errors, 4xx errors, Bytes sent, POST requests and Active connections

We also keep track of your SSL certifications and send notifications to provide warning of expiry

 

For all the parameters we build normal behavior patterns which again are used to define anomalies based on AIMS proprietary algorithms.  The image below shows the actual behavior and normal behavior for requests hitting the DefaultAppPool.  The blue area is the normal behavior AIMS has learned based on historical behavior.  The normal behavior pattern is continuously updated based on experienced behavior.  In the example below we have limited activity on that specific parameter (this is one of our test systems).  It is important to understand that each normal behavior pattern continues to learn and adapt to actual behavior of your system and adjusts to real incidents.

What does an Anomaly warning look like?

An anomaly is both distributed as notification in e-mails and easily availabe in several locations in the AIMS web application.  Below you see a screenshot of the "Default Website".  The screenshot shows data for the parameters fetched for "Sites".  Further you see to the right open errors and warning (anomaly) as highlighted in the screenshot below.  

Clicking the anomaly in the screenshot above (or from the email notification or other screens in AIMS) will take you to the details of the Anomaly as shown in the following screenshot.  The Anomaly shown below is from one of our test systems and shows the initiating component (POST requests) and other parameters showing anomalies at the same time (correlation of all other relevant paramaters) with the actual and normal range.  This warning can easily be used to navigate the topology to understand the location and consequence of the anomaly.  

But you can also go directly to Analytics by clicking "See in Analytics" button which will give you the opportunity to dive deeper into analytics for the specific anomaly as shown in the screenshots below.

1) Select the data you want to chart

2) A new report is generated in Analytics with details that shows the development in the relevant components and performance parameters.

 

But AIMS does not stop at anomalies on one system, we go beyond...

We have above discussed how AIMS tracks performance data on the various artifacts in IIS and how AIMS dynamically develops normal behavior patterns to identify anomalies and correlation of anomalies across performance parameters of components.

But that does not cover the end-2-end scenario where you want to see the cause-impact of anomalies across systems.

So, when you install the AIMS IIS agent you need to do a small piece of manual work - you need to manually define the systems that interact to instruct AIMS to look for correlation of anomalies across those systems.

In the screenshot below (from the Topology overview in AIMS / subsection of image above in this article) we have dragged the IIS into a group consisting of a Windows Server (using the AIMS Windows Servers Agent), a BizTalk (using the AIMS BizTalk agent) and a SQL (using the AIMS SQL agent).  This is a common scenario where these systems have cyclical behavior supporting business processes and transactions flowing between systems and hence systems are depending on each other.  Looking for anomalies and correlation across the systems becomes important.  After you have dragged / dropped the systems into such a group AIMS will automatically start correlating anomaly detection across all parameters on all artifacts.  For the scenario below this could easily be 10.000 parameters that are monitored for anomalies, using dynamic normal behaviors and correlation in real time - without any performance impact.

So, using a set-up as outlined above you will be able to identify performance bottlenecks across your integration middleware, databases, servers and web servers.  You can of course also access all the data fetched and expose the data through intuitive analytics to stakeholders internally or externally to build institutional knowledge to prevent problems, allocate technical staff efficiently and ensure compliance and governance for your business processes.

But we are not stopping with BizTalk, SQL, Windows Server, IIS, HTTP End-point, Filecount, Custom SDK and open API's.  We are already into development of Service bus and Logic Apps and have several new technology support initiatives into the near term roadmap.  Look out for more cool technology support or reach out to AIMS with specific needs and suggestions!