Understanding photochromicity is essential for developing new means of modulating the optical properties and optical response of materials. Here, we report on the synthesis and exciting new photochromic behavior of Nb5+ doped TiO2 nanoparticle colloids (NCs). We find that, in hole scavenging media, Nb5+ doping significantly improves the photochromic response time of TiO2 nanoparticles. In the infrared regime, Nb-doped TiO2 NCs exhibit 1 order of magnitude faster photoresponse kinetics than the pristine TiO2. Enhanced photochromic response is observed in the visible light regime as well. The transmittance of Nb-doped TiO2 NCs drops to 10% in less than 2 min when irradiated by UV-light in the 500 nm range. The photochromic reaction is fully reversible. The physical origin of the high reaction rate is the high Nb5+ concentration. As a donor dopant, Nb5+ builds up a significant positive charge in the material, which leads to highly efficient electron accumulation during the UV irradiation and results in a rapid photoresponse. EPR experiments identify a new defect type from Nb5+ doping, which alters the physical mechanisms available for transmittance modulation. Our new NCs are economic to synthesize and highly suitable for switchable photochromic applications, e.g., smart windows for modulating visible light and infrared transmittance in built-environments.