Microsoft has shared information on how the company optimizes performance on Bing Search by using a system called the Performance Analyzer Service (PAS.)

It runs in a controlled environment of identical machines and is used to measure the performance of front-end code.

PAS runs tests against URLs, also known as scripts, using real browsers in test agents running on the machines. Measurement models for all types of code is not new.

Progress made with national-language processing models (NLP) are often premised on how models perform on benchmarks for various related tasks. In fact, they are intended to serve as measurement models for performance of all types of tasks.

Microsoft’s Bing team uses several tools to measure the effects of the code changes to site performance, before tests go live for users. It also uses time-series dashboards and reports to check the performance of the search engine performance in the real world. If an issue crops up, the team uses an Anomaly Detection tool.

Through Anomaly Detection, the team can check a number of dimension combinations to quickly identify unusual patterns or deviations in metrics, such as spikes or steps in latency or traffic. Anomaly detection allows engineers to identify and isolate issues to specific areas of the system, rather than having to search through a vast amount of data.

Within the Bing division there’s a team dedicated to improving performance. This team’s responsibility is to maintain tools and data used to check on Bing’s search engine and to provide guidance and advice to other Bing team members, giving them the best info to write the best code.

Microsoft has shared information on how it optimizes performance on Bing Search. Here are the tools the team uses to make experiences more useful.