Every once in a while my phone gets very sluggish when I'm multitasking, which is annoying but fine. Is there any way for me to set it so that the Dialer app takes priority over everything else, quickly stealing whatever resources it needs to run as fast as possible?
Also, I am aware of the plethora of task killers available but I'm not interested unless it specifically allows me to set the dialer as a VIP or in some way make sure that the dialer gets first dibs on resources.
Unrooted Moto Droid on 2.2
I don't think there is any way to prioritize threads/apps in Android. Programatically you'd set thread priority here, but it would be your threads in your app. You shouldn't be able to do this against someone else's app unless you had their developer signature as all applications are sandboxed from each other (unless permission is given).
Instead, I would focus on trying to improve overall performance on your phone:
You are right to not use a task killer. The Android OS was built to handle tasks appropriately and you would otherwise waste resources running an auto-kill service every X minutes (because those killed tasks are just going to re-spawn) and potentially cause other problems (killing a task that is writing to the SD card, for instance, could cause file corruption). The only thing task killers are good for are for what you would normally use one for: to kill hung up processes.
The Linux command
renice (use terminal) might be what you're looking for. The
renice program is used to set the CPU and IO scheduling priority of a running process; the higher the nice value (the nicer the program) the lower the scheduling priority, and programs with low nice value (IOW, "mean") have higher scheduling priority.
However, I'm not quite sure how nice values are used by Android framework. In here: http://code.google.com/p/openeclair/issues/detail?id=97 it is hinted that Home apps prevents itself from being killed by setting itself with a very low nice value; and unless Android kernel deviates from upstream linux kernel by way too much, the CPU and IO scheduler should use nice value to prioritize CPU and IO scheduling.
You might need to be rooted and have busybox.
AutoKiller Memory Optimizer (root only) or Auto Memory Manage
It's not specific to the dialer but you can give the dialer a higher priority.