Electric Bike Report’s Testing Process Explained
How We Consistently Measure E-Bike Performance
Here at Electric Bike Report, it is our mission to provide detailed, honest, and objective reviews of each e-bike we test. We want to help you to understand how each product performs and feels, so you can better decide if it’s the right e-bike for your needs.
To do that consistently, we have established a series of standardized, real-world tests that allow us to gather data in a relatively scientific manner. It’s the best we can do without a sealed, climate-controlled lab that would cost millions of dollars to build and staff appropriately (though we’d happily accept donations toward the cause if anyone wants to contribute!). So yes, some human shortcomings are to be expected, and of course your personal results on your own e-bike will vary since there are a host of factors that can alter an e-bike’s performance on any given day. However, this approach allows us to roughly measure any specific e-bike’s individual performance, but also gives us the ability to compare models between various categories, classes, and spec levels. We do our best to keep things consistent, even having certain team members be the dedicated operators on tests where weight is a significant variable.
While our methods have evolved over time (and will continue to), we want them to be as clear as possible. For this reason, we have put together this list of the various trials that make up our testing process, with detailed explanations of each.
Our test circuit consists of a paved 1-mile loop that includes four right turns and a small incline with 30 feet of elevation gain.
First, we make one lap on each e-bike we test with zero assistance from the motor. The e-bike in question is always turned on, but its pedal assist system (PAS) will be set to PAS 0. On this lap, we measure the time it takes to complete, as well as an average of the bike’s speed in miles per hour. We then repeat this process again for each of the bike’s PAS levels.
This test is usually the first we perform on every e-bike, because it allows us to familiarize ourselves with each product in a consistent way. Specifically, it allows us to understand how the bike’s motor engages, how effective its torque or cadence sensor is, and how it handles. It also gives us the chance to see what speeds the bike is capable of in each PAS setting, and informs us what settings to use on our next test.
It’s also important to note that we try to apply a consistent level of effort and maintain a steady pedal cadence throughout the test. We think of it as giving 70% – the type of pedaling you might do on a traditional bike if you were trying to get to work with time to spare, but didn’t want to show up sweaty. Maintaining this consistency makes it easier to separate our own input from that of the motor; we can focus more on how the motor behaves, instead of how much harder we’re able to ride with its assistance.
This trial is actually two separate evaluations performed on our local network of paved multi-use paths here in St. George, Utah.
For each test, we pedal a fully-charged e-bike until its battery expires. One test is performed in the PAS setting that provides maximum motor assistance, which drains the battery quickly while allowing the rider to achieve high speeds. The second test is done in the PAS setting that provides the least amount of input from the motor while still drawing power consistently, which happens over a much longer period of time.
To be fair, this low PAS test can be somewhat subjective. In our experience, many bikes do not provide meaningful assistance in PAS 1, and often don’t help the rider beyond six miles per hour. In these cases, we increase the PAS level until we find a setting that provides an appreciable level of motor assistance. Typically, we stop once we see that the bike is allowing its rider to achieve 8-10 mph with near-constant motor input.
In each test, we measure the distance the bike was able to travel, its average speed, the amount of elevation it gained, and the time that elapsed throughout the process. Most of that data is gathered for optional, anecdotal use in our reviews, but we always report the bike’s mileage.
This test gives us an idea of each e-bike’s minimum and maximum range when used practically. There are many factors that influence the distance an e-bike can travel, and we cannot possibly account for them all, so the results from this test are considered to be a rough estimate. We can, however, use this data to compare to the range advertised by the manufacturer of the e-bike in question, and either reinforce or refute their claims.
Each e-bike we test is brought to a local, paved uphill path called Hell Hole Trail. This course is ⅓ of a mile in length, with an average grade of 12%; it’s seriously steep!
To see how capable they are at climbing hills, we pedal each e-bike up this path in its maximum PAS setting. Additionally, if the model we are testing has a throttle, we take a second trip up the hill using only throttle power. During these evaluations, we measure the bike’s average speed and the time it takes to make the climb.
Hell Hole was chosen because of its extreme nature – it is not a hill that any sane cyclist would choose to pedal up on human power alone, and it pushes even some of the most powerful e-bikes to their limits. Not every e-bike is able to complete the challenge, so we consider any model that conquers the hill to be successful regardless of the time or speed it demonstrates on the way.
With this test, we try to provide further insight to how a bike feels on hills, too. More often than not, e-bikes with mid-drive motors feel great, but their numerical data places them in the middle of the pack. Some of the bikes with the fastest times and speeds to the top might be a blast to ride, but their motors can make some slightly alarming noises on the way. We try to add helpful insight by describing our experiences on top of the data to better paint an accurate picture of hill climbing capability.
For our final trial, we take our test bikes to a nearby parking lot, where we set up a series of traffic cones measured out in 5-foot increments.
Then, approaching the cones, we pedal each e-bike we are testing up to the Class 2 motor-assisted speed limit of 20 miles per hour. We apply the bike’s brakes only once the center of its front wheel crosses the first set of cones, with the goal of coming to a stop as quickly as possible while still maintaining control of the bike. We do our best to replicate the feeling of a near-emergency situation that the average rider might encounter, such as a car suddenly backing out of a driveway a short distance ahead. After the test bike comes to a complete stop, we measure the distance back to the set of starting cones. We repeat this process three times, and then use the three measurements to create an average.
The average stopping distance measurement we end up with allows us to compare the brake performance of different e-bikes with different types of brake systems, different-sized rotors, and different manufacturers. We keep a running average of the performances from every e-bike we have tested, which allows us to see how much a specific model deviates from this, as well as to observe changing trends over time. The fact that we keep our maximum speed to 20 miles per hour also allows us to compare e-bikes regardless of their class.
We perform this test largely to make sure there isn’t an e-bike out there running on unsafe brakes. We’re happy to report that nearly every model we’ve tested has performed safely in this test – and we have been sure to point out those few that did not. As a bonus, we occasionally get to see outliers with much shorter stopping distances than our running average. We try to give extra kudos to those e-bikes in particular!
Reviewer & Team Testing and the EBR Philosophy
The previously mentioned “official” tests are a standard part of the process, and allow us to measure all e-bikes across an even playing field, but they are not the only tests we perform.
Each of our e-bike reviewers spends an additional couple of hours (on average) with each model they write about – both on their own, and during our filming and photography sessions. This provides the time they need to experiment and perform any additional, independent tests they want to try, which may change depending on the features of the bike they are riding. It also gives them the opportunity to think about their experiences and how to communicate them effectively.
Additionally, we take our jobs as reviewers quite seriously. We know how fortunate we are to ride and review e-bikes for a living, and we always try to make sure we are being fair and objective. That means doing our best to eliminate any personal bias, especially if it skews negatively. We love sharing our excitement if a bike proves to be something special, but in cases where we might personally be less thrilled, we realize that A) not everyone shares our opinions; and B) each bike we test has positive features and is likely to appeal to someone.
Additionally, we don’t want to pull punches either. If we can’t provide concise critique to the brands then changes that would benefit the product in future iterations can’t be made. Also, we don’t want to gloss over any flaws that could lead to one of our audience purchasing a bike we reviewed only to be dissapointed if we didn’t disclose a not-so-welcome element of the bike they would have to live with.
In other words, it’s a balancing act that we always keep in mind when evaluating a bike’s performance.
In the spirit of objectivity, we make sure that each bike is ridden by several members of the EBR team with various experience levels, ages, heights, weights, etc. This happens through both the various “official” tests, and also in the previously mentioned personal time. While each e-bike has a lead reviewer, the final written and video reviews incorporate the thoughts and opinions of many diverse individual riders. Our review process also includes a group discussion to break down every specific e-bike’s strengths and weaknesses, often in relation to its cost. This approach helps us to realize our goal of providing a broad but detailed perspective on each bike, so we can help each member of our audience find the right model for them.
A Word of Thanks
We want to take a moment to thank you for your interest in our process, and for supporting our efforts here at EBR! With the detailed explanations above, we hope it is clear that we strive to be the most thorough product testers in the ever-growing world of e-bikes. We truly could not realize that goal without the continued backing of readers like you. As things evolve, we hope to see more and more people join the e-bike revolution, and to encourage that, we promise to continue providing useful, honest, and thoughtful content!
We look forward to seeing you on the trails. Happy riding!
Leave a Reply