You seem to be lumping two groups of people who are rather different: There is type 1) the fans who complain about AOs because they believe they are not as good on average, and type 2) the fans who are dismissive of AOs because they don't impact the main storyline significantly.
I sympathize with the type 1s some. The early AOs were pretty good, but the modern ones generally aren't as good as Gosho's mysteries. There are some exceptional standout AOs, and I generally like the mystery tours because they show interesting cultural tidbits, but otherwise I don't feel like AOs are worth my time. I am a type 2, because I hardly have time to deal with anything but the main manga canon.
If you are in DC mainly for the ride, manga canonicity vs anime/movie canonicity doesn't really matter. If you are in DC because you want to play detective and uncover who the BO boss is and the mystery behind the BO's true plan, then you have to be picky. In a series like this, a fan who watches and enjoys everything, AOs included, and one that specializes in manga canon only are equally "True".
As Jimmy said, canonicity matters to theories. In a mystery series, facts are important and they have repercussions. The only way to guarantee you have accurate facts is to only get them from the person who knows the true answers to everything in full detail: Gosho Aoyama. The anime has historically screwed over anyone trying to solve the arc mysteries. There were continuous mistakes and alterations: from creating plotholes that made it impossible for Conan to learn Gin and Vodka's codenames, to missing clues that made it impossible for an anime watcher to solve Akai's faked death. It really is in only the past 4-5 years that the anime has been consistently right on clues and matters of the plot. In general, only the manga and interviews and whatnot are guaranteed to be accurate without enlarging or distorting things for dramatic effect.
The other issue is that it can take year+ to fully catch up in the show. DC is freaking long. The manga is nearing 1000 chapters. If you streamed the anime and movies and specials continuously, 24 hours a day, skipping all openings and endings, you would have to stay awake ~20 days straight to watch it all. If you decided to spend an hour watching DC every night to catch up, it would take you a year and four months. If you were paid $8 an hour to watch DC, then when you got current, you will have earned, ~$3800 dollars. Assuming you have school or a job, that's not an easy time commitment to make. Many people shortcut by ignoring AOs because they want to get current quickly. It's dismissive of AOs, but a reasonable and understandable move.