Why no data on shootouts and bike tests?

MXA does offer rider lap times on different bikes during tests. Top speed is pretty irrelevant as you can gear any 450 to go faster than the other one.  

How about speed in each gear? Or 15mph to 35mph times? 

”it pulls forever in 2nd” wouldn’t data to back that up

Interesting especially if  its mph was lower than all the others? Sure it pulls forever because it doesn’t make enough power to quickly run through the gear… is that really a positive?

Without data it’s a giant circle jerk of advertising.

Speed in each gear is solely related to gear ratios and tire size. The engine has nothing to do with it, until you get up to a speed that's power limited by air resistance and friction, but that will be deep into fifth gear. You can calculate all this with a spreadsheet (I have one here if you want: https://www.luxonmx.com/resources.html). 

15-30 mph times are much more relevant as that's related to the above and the performance of the motor. Doing this isn't as simple as it sounds to have a properly controlled test with meaningful results, though. 

Regardless, all of this just adds to the already daunting task of organizing and reporting the tests as they do now. They're limited in resources, time, and budget, so while additional data would be great, it's a lot to ask. Vital posted a podcast that covers it pretty well here: https://www.vitalmx.com/videos/features/are-shootouts-still-relevant-what-can-we-do-fix-them-vital-mx-testing-and-tech-show 

Yeah it’s impossible to add anything objective  of value so we can just keep grading it al subjectively and trust us bro….

And that was a couple ideas while typing with my freaking thumbs on a tiny ass screen. Ya think maybe I have more ideas if I were properly motivated, let alone ya think maybe the people getting paid to do this shit could maybe try a little harder than this??

Of all people, I would be one to argue for more data. I absolutely understand the desire for it and also want to see it, but I also realize the limitations and difficulty in obtaining good, presentable data. Outside of weights and sometimes dyno plots, where have you seen any data in shootouts? Or in any product review for that matter? They're primarily all subjective, and mostly because they have to be given all the limitations. 

Sometimes lap times are presented, but that's pretty questionable data as there's so much variance lap to lap, bike to bike, time of day, etc. Look at pro racer lap times at a national. Their times vary multiple seconds lap to lap, and that's all top pro riders, on the same bike, over the course of only 34 minutes. Sure there are some other factors at play there, but if there's that much variance there, how much is there over the course of a full day with amateur riders? It's going to be a much larger variance! So how telling are those lap time comparisons? Not very useful. 

And to address the part of your reply you just edited:

I'm not crapping on your ideas, just presenting some insight as to why it's not so feasible to do those things. It's really easy to complain about things, but really difficult to come up with a good solution. That's why is hasn't been done. If you think of a good idea, that's reasonable to implement, please share it. 

I understand what you are saying but I'm not sure most of that info would really translate to much value on a track. It would just be for entertainment than anything, which is fine. Just like how The YZ250F is considered the best 250 engine by most people, even though it has never shown to be anything special on the dyno charts. It's just how the package performs and some things are hard to quantify.

I value someone like Keefer or ML who really know how to explain what a bike does. If I can have that info, as a package, then I can typically use that to choose what bike will fit my style. I focus on those details (what they are experiencing in turns, how the power comes on etc) more than the results anyways. Granted, what they feel might not always be the same for everyone but I still think a good test rider can describe a bike's track mannerisms pretty well and that to me the info that matter's most in test / shootouts.

There'd be a few here that will remember the first few years of (US) Dirt Rider Magazines bike tests.

They had Full engine Dyno Charts,  oscilloscope shots of rear shock dyno runs, and full Leverage ratio charts  and spring curves of rear suspension set ups.

And, much more.

It must have taken a lot of time / man hours to do so, and they stopped it after a few years, I guess because of that.

** The shock leverage ratio / spring curves, and shock dyno charts were incredibly interesting. So Wildly different between manufacturers and models , at that stage. And, the extreme rising rate ones, were always the duds.  And cripes, when you'd see the combo of a nice rising rate  created by the simple shock geometry on the last model Husky twin shock bikes, plus the multi rate springs, and , the ITC internals of the Ohllins, you could see why they worked so well. But, the trend towards single shocks, and their  - Huskies Ohlins - lack of external damping adjustment, and, the twin shocks being easily damaged - especially with the shock eyes on the outside of the swingarm - had Husky give in, and go to a single shock set up.

 Separate from the dirt bike side , I made quite a few dollars alone, many years ago, on making new linkages for  various model / brand Superbikes, that had linkages that were made to contend with the needs for road use / pillion use. I'd go back to near ( but not quite) straight / linear ratios for a few bikes - especially the ZXR750s. 

So much is dependent on rider and things that you can change, like gearing. Drag races are not just a measure of  bike performance, as rider skill is a factor, as is weight. So the most objective things you're going to get are things like weight, seat height, dyno number. Even wheelbase will change depending on the size of the chain and sprockets, which affects where the rear axle is positioned. I agree with the person who said roll on power would be interesting (something like 15-30 mph acceleration). Use a radar gun and the same rider for each run. 

Given the limitations, I think the subjective reports are, for the most part, the best we can hope for - especially if that includes how a bike responds to different kinds of setups (gearing, suspension settings, etc), which MXA typically reports. I remember Vital having very detailed write ups for the shootouts. Keefer has great info and it sounds like he is very generous with his time and feedback. 

And yes, I do trust MXA, ML and Keefer to be doing their best. They've been doing it for a while and know their shit.  I get wanting more data, but to me, when I read someone saying  "ya think maybe the people getting paid to do this shit could maybe try a little harder than this" it doesn't come off particularly well. I have a hard time believing that these guys do things the way they do because they aren't trying - that they're just mailing it in and don't give a shit about the consumers who buy the bikes. They come across as exactly the opposite of that. What's more likely: MXA, Keefer, and ML couldn't be bothered to try, or what is being asked for isn't feasible?

I’m pretty sure Karel Kramer or Tom Webb said in a Matthes podcast that Dirt Rider got some new bean counters who kaboshed the detailed tech specs. Charlie Morley assumed reader feedback would overwhelmingly favor bringing them back.

They didn’t hear from any outraged readers. 

Yes, it must have cost them a small fortune to do those Test reports / Specs - I was amazed the Bean Counters allowed it to go for as long as it did.

It was very interesting while it did exist though. 

Just run a Draggy on each bike on pavement. Problem solved with real world data related to performance.

I think part of the problem is that you need to first determine what exactly it is you’re trying to test, and then work backwards from there.  On a motocross track what are we realistically concerned about?  A few things come to mind:

1. Suspension performance

2. Acceleration out of corners

3. Braking performance coming into corners.

Now, let’s just take acceleration out of a corner as an example (notice I said out of a corner, not just pure acceleration).  If we want to know which bike does this best would a drag race on pavement be a good representation?  Not really, because this tells us nothing about how the bikes geometry, suspension performance, and motor character may effect things like traction out of a rough and rutted corner.  All the power in the world is doing no good if it’s not getting to the ground, or if the motor is too aggressive and is just lighting the tire up rather than driving forwards.  Both of these factors would be invisible in a drag race, but would have a huge effect on the behavior we’re actually worried about under normal use cases.  

The same thing can be said for the other examples I gave.  I would love to have more hard data in bike tests, such as in depth chassis measurements like they do for mtb, but the reality is most things are at least a little bit subjective and even inconsistent when it comes to evaluating real life performance.

Nobody cares enough to figure out the solutions and the consumers are ok with subjective nothingness so there’s no reason for them to try🤷‍♂️

Will this reveal anything that a dyno run of each bike wouldn't? 

It will add in the human element, which will lead to loads of inconsistencies: reaction times, shift point variance, etc. will all skew the results. Some riders may gel more with a Honda clutch than a KTM clutch, but that doesn't mean the Honda is better because that one rider preferred the clutch. So you'd need multiple riders to filter that out. And multiple runs to weed out the outliers due to bad reaction times, etc. Let's say 5 riders, 10 runs each bike, and 10 bikes. That's 500 runs to preform, keep track of, tally up, analyze, etc.  If it takes only one minute to preform one run and set up for the next one, that's over 8 hours non stop. 

Dyno runs have their limitations, but they're nice in that it largely removes the human element, gives more data, and happen quickly. 

And to be clear, I'm not simply trying to crap on your idea, I'm just showing that it gets a lot more complicated when you're trying to execute it properly and get meaningful results. 

Get an accelerometer and measure vibration through the rpm range at certain touch points

Measure the rider triangles from a reference point. 

Post the center of gravity which the mfg will probably share

Not to mention any speed, braking, or kinetic testing that would be of interest as already mentioned. 

Shootouts are mediocre because they’re not “shootouts” they are just in house advertising.

-potentiometer on teh suspension with a smoothed graph or distribution chart would be cool. i.e. where does the bike sit in its travel throguhout a lap compared with another. 

-average throttle position, with the implication being comfort/traction youll see more throttle, assuming similar torque curve

-maybe steering angle or yaw or lean angle

its too bad moto isnt more popular in germany. they do the best most clinical tests....

I’ll bet the speed in 2nd gear reveals that the yz is geared lower than the others significantly. So the power house yz isn’t just torquey or responsive it’s just geared much more aggressively than the others. Gear a Ktm down by 4 teeth and I bet it’s snappy too… there’s things to learn but instead we’ll just get, this one rips broooo, totally gnarly bro, by the way thanks to decal works for making all the bikes look totally rad broooooo

MXA always discusses gearing. When KTM went to 14/51 gearing on the 250 MXA suggested changing the countershaft sprocket to a 13 which is the equivalent of gearing down 3 and a half on the rear. There’s trade offs to changing gearing though, which they address. 

https://motocrossactionmag.com/mxa-race-test-2017-ktm-250sxf/
 

The dyno’s now days are advanced to give a decent amount of data.  Tells me what I need to know.  It would be interesting to see some old fashion drag races.  A little more info, not necessarily data on ride ability as Beta refers to it for older guys/guy who just want to make a few laps at the track.  Not everybody is top class rider looking for more speed.

MXA says that a +1 rear sprocket improves all the data. 

Results wise? No. A new type/method of data and food for thought for the consumer? Yes.

As someone who still subscribes to paper moto mags, the dyno charts presented are barely legible given the size of print, and how close all the machines are when it comes to power curves. 

As a fellow engineer, I can appreciate your thought process when it comes to poking holes in someone's idea, and you are absolutely correct when it comes to the human element, number of runs etc to weed out any anomalies...But at the same time this would be a test report for a consumer base who believes that decreasing the torque value of an engine mount fastener a small amount can "increase" chassis "feel".

Throw a disclaimer in the sub text below the chart, and GEMO.

And PS - on a small NA engine, dyno runs have their own variability too! PCs dyno (commonly used by magazines for example) is in an open area of their garage (obviously not a CTH room) so atmospheric conditions (air temp throughout the day) affect output, as does engine temp, oil temp, number of runs it has had on it on the dyno before the hero run, heat soak, air flow conditions, etc etc. See I can poke holes too haha. But I try to add value and solutions to conversations, not just have a problem for every solution.

Do you really think the realistically 10+ hours the drag runs would take would be worth it for the end result? That's more than an entire test day devoted to something that reveals dramatically less than a dyno would in only a couple of hours...

I've said it multiple times over in this thread, I'm not simply poking holes in ideas to poke holes. I'm providing context and details as to why something that sounds like a simple idea is actually very difficult to implement and infeasible. If I (or anyone else) is able to poke huge holes in an idea, it's simply not a good idea. 

You've shown that you can poke small holes in dyno testing, but every one of those holes also exist in the drag race, in addition to the bigger holes I mentioned earlier. You can poke holes in any idea if you try hard enough, but if they're small relative to the result (dyno runs), then that's a much better idea than the massive holes relative to the minimal data revealed by the drag runs. Everything is a trade-off, but in this case the best option is overwhelmingly clear, which is why magazines do dyno runs and not drag races. 

How long does it take to do 6 dyno runs on motocross bikes? 

1.5 hours each? Between pulling the rear wheel, putting on a street tire, loading the dyno, baseline for wheel speed calculating RPM calibration, multiple pulls, unloading, re-installing a knobby tire. Now that I typed all that out, 1.5hrs is probably generous, that's 9hrs right there...

And yes I do. Its something new and different, try it. If it doesn't work, go back to ole' reliable dyno tests.

Also FWIW, when I originally suggested a Draggy, in my head I envisioned it as a singular gear roll on from one speed to another (ie. 20-40mph in 4th gear) to minimize variables, which I guess I didn't explicitly state.

If it best option for data that is "overwhelmingly clear" doesn't correlate to real world in use input (Some bikes are big HP makers, but don't feel that way on the track if its a very peaky poweband for example) is it really the best option?

I'd guess 30 minutes per bike tops by someone experienced at it. Wheel swaps can be done in parallel with the dyno runs by another person. Something like 5 hours for 10 bikes vs. 10+ hours if you want an apples to apples comparison to the drag run example I posted. 

And even if I'm way off and it takes an hour per bike, that's still the same amount of time as the drag runs. I'm actually blown away that as an engineer you'd prefer wildly variable drag runs over controlled dyno pulls, resulting in only two data points (time and speed) vs a full plot of torque vs RPM. So, we're going to have to agree to disagree on that one!

How can we be so sure they would be wildly variable when no one has tried it?

I'm not speaking to this from an solely an engineering perspective. I'm proposing getting away from the whiteboard and just talking about theoretical things and potential problems, and instead trying them and seeing what comes from it, I'm sure some experiment or test you have done at some point in your career has gone the opposite way of your initial theory.

The average motocross consumer IMO doesn't care about dyno curves, they care about peak HP numbers, and torque is for wrenches. Something simpler such as the metaphorical 60-130mph test (for dirt bikes) could sway their decision making when it comes to motocross bikes. 

Also, by your own admittance - you "guess."

Trying things and experimenting to see what comes from it is far more valuable than just talking about problems and eventually talking people out of even trying.

Anywhoo, pleasure debating you Billy. I need to get back to work now ha.

1. If error is consistent across testing it can be accounted for and to some extend disregarded. So yes, personally (also as an engineer) I’m fine with an experienced rider ripping drag pulls if he’s intelligent and intentional enough to run them the same. When minimal I don’t see human error factors as being a problem for human applications like this. 

2. Dyno runs are honestly the most useless thing on earth for dirt bikes I don’t think they’re worth anything more than bragging rights and figuring out where a bike makes good torque. 

Perhaps “guess” was the wrong wording. While I’m no dyno expert, I have spent multiple hours on one, with multiple vehicles. It was nowhere near an hour and a half to get up and running and make multiple consistent pulls. 

Evaluating the feasibility of processes/designs/testing/etc. via professional thought experiment is one of the primary jobs of an engineer. It's what we do, on a daily basis, over and over again. There is often use in physically testing ideas, proof of concept, etc., to aid in that thought experiment when the scenario is complex. A drag race is not one of those. It can easily be eliminated via the thought experiment I’ve outlined here. 

To me at least, it's obvious that drag pulls would be a huge time suck, providing unreliable and rather useless data, particularly when compared to the wealth of data a dyno run provides, more accurately, and in less time.

I cannot believe you guys are arguing with Luxon’s common sense. Data points from a 20mph 4th gear roll would be useful for the inner city grudge racer, not your average MX rider looking to purchase a new machine.