Plummer's family is full of professional baseball players. His father, William Lawrence Plummer, and uncle, Earl "Red" Baldwin, played in the Pacific Coast League in the 1920s and 1930s. Plummer's grandson, Conner Menez, is a pitcher for the San Francisco Giants.