Communication processors are processors with specific optimizations to support communication sys-tems. Communication processors exist in a wide variety of forms and can be categorized based on the communication system, such as wired or wireless and based on the layer in the communication system, such as the physical layer, the medium access control layer or the network layer. Communication processors can be further categorized based on the application, such as audio, video or data and the end system requiring the communication system such as a laptop, a cell phone or a personal computer. In this book chapter, we present a brief outline of the different types of communication processors and the need and requirements of each of these processors. However, we will focus on the challenges in the physical layer design in communication processors with the increase in data rates, increase in algorithm complexity, need for flexibility to adapt to different protocols and environments, need to optimize over varying con-straints such as area, power, performance and the need for supporting multiple interfaces, devices and applications.