Bring a pot of salted water to a boil. Boil the potatoes until fork tender but not mushy, about 20 minutes. Drain, then rinse in cold water, and cut into quarters. Place the warm potatoes in a large bowl and toss with onion, celery, dill and parsley.
In another bowl, stir together mayo, celery seed, cayenne, vinegar and lemon. Check seasoning.
Add the dressing to the vegetable mixture. Toss gently to coat taking care not to mash the potatoes. Season with salt and pepper.