We present a detailed Raman study of defective graphene samples containing specific types of defects. In particular, we compared sp3 sites, vacancies, and substitutional Boron atoms. We find that the ratio between the D and G peak intensities, I(D)/I(G), does not depend on the geometry of the defect (within the Raman spectrometer resolution). In contrast, in the limit of low defect concentration, the ratio between the D′ and G peak intensities is higher for vacancies than sp3 sites. By using the local activation model, we attribute this difference to the term CS,x, representing the Raman cross section of I(x)/I(G) associated with the distortion of the crystal lattice after defect introduction per unit of damaged area, where x = D or D′. We observed that CS,D=0 for all the defects analyzed, while CS,D′ of vacancies is 2.5 times larger than CS,D′ of sp3 sites. This makes I(D)/I(D′) strongly sensitive to the nature of the defect. We also show that the exact dependence of I(D)/I(D′) on the excitation energy may be affected by the nature of the defect. These results can be used to obtain further insights into the Raman scattering process (in particular for the D′ peak) in order to improve our understanding and modeling of defects in graphene.