Statistical Significance
Statistical significance is a conclusion reached in an A/B test when the data shows that the observed result is very unlikely to have occurred due to random chance. It is typically determined by calculating a p-value. If the p-value is below a predefined threshold (usually 0.05), the result is considered statistically significant, giving you confidence that the change you made (e.g., improving page speed) was the actual cause of the measured improvement (e.g., a higher conversion rate).
Related web performance terms
- Search Engine Optimization (SEO)
SEO is the process of improving a website's visibility in search engine results pages (like Google). The goal is to rank higher than competitors for relevant search queries to attract more organic (free) traffic.
- Server Response Time
Server Response Time is a component of Time to First Byte that measures how long the server takes to process a request and start sending back a response, excluding network latency.
- Server Timing
Server Timing is a browser API that allows the backend to communicate performance metrics about its own internal processes (e.g., database query time, time to render a component) to the browser via HTTP headers.
- Server-Side A/B Testing
This is a performance-friendly method for running A/B tests. Unlike Client-Side A/B Testing, the decision of which variation to show a user is made on the Origin Server.