Nanomaterials as reinforcement to enhance the properties of the composite and extend its potential applications have always shown great interest. As 1D materials, aligned carbon nanotubes (CNTs) reinforced composites could show different mechanical properties in different directions. In this paper, the frictional anisotropy of oriented CNTs/rubber composites at both macroscopic and atomic scales was investigated. Both physical experiments and molecular dynamics (MD) simulations were carried out to measure the coefficient of friction (COF) of rubber incorporated with CNTs oriented in different directions. The macroscale and nanoscale results qualitatively show good agreement. The COF and wear rate of CNTs-z/rubber are both lower than that of CNTs-x/ and CNTs-y/rubbers. That is, CNTs perpendicular to the sliding interface could further decrease the COF compared to those that are in the plane parallel to the interface. This may be due to the weaker adhesive ability, smaller real contact area, lower temperature, and higher anti-shearing ability on the interface. This work provides guidelines for the extension of the application and control for the CNTs reinforced composites.