This is a loaded question. There are elitist in absolutely everything. What you are experiencing is likely due to the percentage of sales that dealership does. Also the possibility of a sale. I don't have any hard data, but I would imagine street bikes account for a major chunk of a dealers bottom line as compared to motocross bikes. Street bike riders come in all shapes and sizes. It is also easy to have a mid life crisis and go get a Harley. Grab a helmet, boots and some sunglasses and you are set to go. Motocross requires a lot of gear, a truck and a reasonable understanding of what it takes to "go" dirt bike riding.
The dealer needs people who cater to the customers that contribute the most to their bottom line, simple as that.
As far as the rest of the discussion... After a decade and a half of racing everything from motocross to desert to arenacross, then buying a Harley recently, its very true. I was never fast in the grand scheme on a dirtbike, I wouldn't say slow though...
What you really have to think about is how you ride a dirt bike. Even when I was out on practice days or carrying the mail across the desert, I liked to push it, go fast and try to be faster than the buddy you were out with that day. When you are out on the street, you have to obey laws, so for the most part you have to ride within those limits. Those who have saddle time on the road probably have minimal time "pushing" it.
Not to mention a motocross bike is infinitely harder to ride than a street bike. I'm not talking about that niche practice day sport bike guy who is dragging knees... I am talking about the average rider. You have to exert a bunch of energy to go even remotely fast, and generally control the machine more. Once you have done that on a 450 around Glen Helen, hoping on a Harley and cruising the Ortega seems like a cake walk.
- When the Going Gets Weird, the Weird Turn Pro