A data sample containing top quark pairs (t¯t) produced in association with a Lorentz-boosted Z or Higgs boson is used to search for signs of new physics using effective field theory. The data correspond to an integrated luminosity of 138 fb−1 of proton-proton collisions produced at a center-of-mass energy of 13 TeV at the LHC and collected by the CMS experiment. Selected events contain a single lepton and hadronic jets, including two identified with the decay of bottom quarks, plus an additional large-radius jet with high transverse momentum identified as a Z or Higgs boson decaying to a bottom quark pair. Machine learning techniques are employed to discriminate between t¯tZ or t¯tH events and events from background processes, which are dominated by t¯t+jets production. No indications of new physics are observed. The signal strengths of boosted t¯tZ and t¯tH production are measured, and upper limits are placed on the t¯tZ and t¯tH differential cross sections as functions of the Z or Higgs boson transverse momentum. The effects of new physics are probed using a framework in which the standard model is considered to be the low-energy effective field theory of a higher energy scale theory. Eight possible dimension-six operators are added to the standard model Lagrangian, and their corresponding coefficients are constrained via fits to the data.