All lower case. The plural form is justices of the peace. Oddly, the plural abbreviation is JPs.