This article presents the results of experimental measurements of the physical properties of new environmentally friendly bio-based composite building materials containing hemp shives bonded with a magnesium binder. Some of the tested materials contained an admixture of phase change material (PCM) of variable proportions in the binder to increase the heat capacity of building elements (walls), which can positively affect room temperature regulation. Densities and porosities are key parameters describing building materials, directly affecting mechanical, acoustic, and, most importantly, hygrothermal properties, including thermal conductivity, water vapor permeability, water absorptivity, and sorption curves. The experiment was carried out for ten different samples of bio-based building composites, differing in the bulk density obtained during the manufacturing process and in the PCM proportion. As part of the experiment, true density tests were conducted on a helium pycnometer. Then, the geometric densities of the tested materials (which may differ from the bulk density obtained during production) were measured using the Archimedes method, making it possible to obtain the total, closed, and open porosity values. Tests were also carried out for selected traditional building materials, such as red brick and autoclaved aerated concrete, to compare the results obtained.