Measurements of azimuthal angle and transverse momentum (pT) correlations of isolated photons and associated jets are reported for pp and PbPb collisions at sNN=5.02 TeV. The data were recorded with the CMS detector at the CERN LHC. For events containing a leading isolated photon with pTγ>40 GeV/c and an associated jet with pTjet>30 GeV/c, the photon+jet azimuthal correlation and pT imbalance in PbPb collisions are studied as functions of collision centrality and pTγ. The results are compared to pp reference data collected at the same collision energy and to predictions from several theoretical models for parton energy loss. No evidence of broadening of the photon+jet azimuthal correlations is observed, while the ratio pTjet/pTγ decreases significantly for PbPb data relative to the pp reference. All models considered agree within uncertainties with the data. The number of associated jets per photon with pTγ>80 GeV/c is observed to be shifted towards lower pTjet values in central PbPb collisions compared to pp collisions.