To give the system its due, mathematically it appears very sound. The goal of "extending the sweet spot" and ironing out some of the wild disparities in ability between a random mix of characters and monsters, especially at high level, seems to have been met admirably. A 1st level character and a 10th level character are on a wildly different playing field, yes; but a 10th level fighter, a 10th level wizard, and a 10th level monster are all roughly in the same ballpark. (Well, sort of — there are different flavors of 10th level monster which are intended to be "harder" or "softer" depending on their role in the encounter. More on that later.)
The best implementation of this is the section on "Situations the Rules Don't Cover," which I've seen referred to in various places as "Rule 42" (presumably because it appears on page 42 of the DMG). In a nutshell, this section is about how to handle all those wild and woolly things players are always pulling out of their ears: "I flip the table over on him!" "I dash up the banister!" "I throw a chicken in his face!" When confronted with one of these, the GM makes a choice: is it like a skill check, or is it like an attack? If it's like a skill check, should it be easy or hard? And if it turns out to be like an attack, should it do a little damage or a lot of damage?
Once these decisions are made, the GM is provided with a chart that gives level-appropriate ranges for the difficulty numbers (or AC of an improvised attack), and damage ranges, something like this:
PC level: 1 - 3
Skill DC easy: 10
Skill DC medium: 15
Skill DC hard: 20
Low Damage: 1d6 - 1d8
Medium Damage: 1d10 - 1d12
High Damage: 2d8 - 4d6
I don't remember the exact numbers, I just pulled these out of the air for illustration. The point is, this table is carried through from level 1 - 30, and is also used as part of the basis for creating custom monsters elsewhere in the book. The numbers have been very carefully calculated to reliably challenge-but-not-overwhelm the PCs from beginning to end, instead of having everything be deadly at low level, a good mix in the middle, and then a coin toss between "cakewalk" and "TPK" for every encounter after 11th level, which is a problem that has bedeviled D&D from the Gygax/Arneson days.
Unfortunately, there's a drawback to this careful balance, which is that it produces weird game-rule artifacts that don't make sense in the fictional reality of the game but are required to make the math work. For instance, monsters have a "magic threshold" based on their level, which is the maximum bonus they can derive from a magic item. A monster of 1st - 5th level has a magic threshold of +0, 6th - 10th a threshold of +1, etc. (Again, I'm not sure of the numbers, as I don't actually have the book in front of me.) So if by some weird quirk of fate, Joe Goblin has the Axe of the Dwarfish Lords fall at his feet, he gets a bonus of ... +0. But if Joe Goblin manages to claw his way up to 6th level, suddenly he gets +1 from this crazy powerful artifact. The Axe isn't any more or less magical than it was before — but by passing that magical 5th level barrier, Joe Goblin suddenly gets a boost he wasn't getting before.
Of course, the game designers' response would probably be, "Who gives a goblin the Axe of the Dwarfish Lords and then bothers with the math?" Which is a fair cop, but I'm just using this as an example. Monsters and NPCs are mechanical abstractions in 4E; not as bad as the old days when they didn't even have ability scores, but still playing by their own rules, which has always irked me. Their hit dice, attack bonuses, etc., are not a function of what kind of creature they are (humanoid, dragon, undead, etc.) but by what their role in combat is (artillery, brute, minion, etc.). It's not necessarily a bad way to do things, but what I don't like about it is that RPGs are already once-removed from the setting by having a ruleset in the first place. For that ruleset to be defined by "the needs of the math" rather than by "the needs of the setting" removes it even further. I suspect this is one of the major reasons people are shrieking about 4E being "like a miniatures game" or "like a videogame" — it is more "gamey" than 3.x was.