Pocket PC (touch screen): Just Tap+Hold your Stylus on any card or category you want to delete or rename. Using Tap+Hold with most programs on your Pocket PC will show you what you can do with any of their objects.

Smartphone (no touch screen): Simply select the item you want to delete, and pick Menu -> Edit -> Delete.