The specific recognition of peptides, which we define to include unstructured regions or denatured forms of proteins, is an intrinsic part of a multitude of biochemical assays and procedures. Many cellular interactions are also based on this principle as well. While it would be highly desirable to have a stockpile of sequence-specific binders for essentially any sequence, a de novo selection of individual binders against every possible target peptide sequence would be rather difficult to reduce to practice. Modular peptide binders could overcome this problem, as preselected and/or predesigned modules could be reused for the generation of new binders and thereby revolutionize the generation of binding proteins. This minireview summarizes advances in the development of peptide binders and possible scaffolds for their design.