Preheat oven to 400°F.
Arrange potatoes in single layer on large rimmed baking sheet. Drizzle with oil; sprinkle with rosemary pieces, garlic, and salt; toss to coat. Roast until potatoes are tender, stirring every 15 minutes, about 45 minutes total. Mix in capers. Roast until flavors blend, about 5 minutes longer. Season with pepper. Transfer to bowl. Garnish with rosemary sprigs.