Rapid Field Selection
In addition to dragging fields from the "Fields" pane, it would be nice to click into a visualization area (such as "Axis", "Legend", "Value", etc.) and start typing. Each area would become an autocomplete box, and populate a field name based on what you have typed so far.
Additionally, you could move between slots (perhaps from "Axis" to "Legend") by hitting tab or enter. This keystroke would confirm the autocompleted value currently in progress and move you to the next field.
The end result would be the ability to click into "Axis", type the first characters of your field name, hit enter or tab, do the same for "Legend", and create a full chart in seconds.