Load Tests

Florian Rappl, iQuest

Load Tests

Evaluating System Performance

Cover Load Test

Florian Rappl

Solution Architect

  • IoT / Embedded Computing
  • .NET Communication Systems
  • Web Technologies

Writer and consultant

  • Microsoft MVP for C# / Development Tools
  • Active contributions to open-source project s
  • Articles for CodeProject, SitePoint, tuts+, ...


Indian Truck Overloaded
Indian Truck Fail

Why does it matter?

Time Perception
0.1s Users feel their actions are directly causing something to happen without being interrupted
1s Users feel they are navigating freely and stay focused on their current train of thought
10s Users feel a break of the flow, get impatient while waiting and leave the site
Jacob Nielsen, Power of 10: Time Scales in User Experience, 2009

Why does it matter?

  • 49% of online shoppers expect load times of <=2 seconds
  • 50% of online shoppers abandon page if load time is > 2 seconds
  • 42% of men and 35% of women won't come back after experiencing slow load times

Further Reading: latencytipoftheday.blogspot.de

Types of Load Tests

Load Test Types
Normal Load Distribution

Where does it fit in?

Testing Pyramide

Test before it is too late...

Test Before it is too late


  • Status (idle, working, offline, ...)
  • Throughput (# of requests)
  • Max. queued requests
  • Internal IO usage (disk, network)
  • Processor usage
  • Custom counters
  • Load distribution

Machine Status

Load Test Signaldiagram
Beware of Averages

The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty; Sam L. Savage, with illustrations by Jeff Danziger.

Spike Detection

Network Spikes

Result Interpretation

Load Test Result Interpretation

Rule of Thumb

Always try to be deterministic!

Also it helps to have some kind of logging in order to validate any possible assumption.


Load Test Execution (Server)

Load Test Execution Server

Load Test Execution (Client)

Load Test Execution Client

Load Test Execution (Plan)

Load Test Execution Plan
  • Arbitrary action
  • Could be also just a timeout / idle

Load Test Execution (Flow)

Load Test Execution Flow

Main Challenge

The load test needs to scale at least as good as the target.

We require flexibility even though agents need to run without supervision.

Targets may be on-promise, in the cloud, or mixed.

There is more than HTTP or RESTful APIs.

Available Commercial Solutions

  • AppDynamics
  • New Relic
  • Pulseway
  • Dynatrace
  • VS Load Test

AppDynamics Dashboard

AppDynamics Overview

Available FOSS Solutions

Load Test FOSS Solutions

Image by Dr. Alexander Wert and Dr. Christoph Heger, 2016.

Heinrich Logo


Heinrich Architecture


Heinrich Lifecycle

"description": "My test description",
"resources": [{
	"provider": "azure",
	"options": {
		"authorization": { ... },
		"deployment": {
			"region": "West Europe",
			"storage": { ... },
			"cloudService": { ... },
			"serviceBus": { ... }
"configurations": [ ... ]

"workers": 8192,
"threads": [{
	"type": "simulator",
	"setup": { ... },
	"execution": [
		"Delay({ time: `${workerId * 5}s` })",
		"RandomIdle({ min: '50ms', max: '1s' })",

Using Azure

Azure Load Test Scaling

Using Azure

Azure Load Test Limitations
Azure Load Test Limitations


  • Integrate load testing in the dev cycle
  • Collect the right metrics
    • of the target
    • of the load test itself
  • Be careful with result interpretation
  • Use a load test solution that scales


Thanks for your attention

  • Feel free to contact me