In this study, the mechanical behavior of epoxy/carbon nanotubes (CNTs) nanocomposite is predicated by a two-scale modeling approach. At the nanoscale, a CNT, the interface between the CNT and the matrix and a layer of the matrix around the CNT are modeled and the elastic behavior of the equivalent fiber (EF) has been identified. The CNT/epoxy interface behavior is modeled by the Park–Paulino–Roesler (PPR) potential. At the microscale, the EFs are embedded in the matrix with the extracted elastic properties from the nanoscale model. The random pattern has been used for the dispersing of EFs in the representative volume element (RVE). The effect of CNTs agglomeration in the epoxy matrix has also been investigated at the micro level. The Young’s modulus of the nanocomposite was extracted from simulation of the RVE. CNT/epoxy nanocomposites at four different volume fractions were manufactured and the modeling results were validated by tensile tests. The results of the numerical models are in good agreement with the experiments and micromechanics theory, and by considering agglomeration of CNT in the model, the modeling results match with the experiments