By the reduction of channel spacing in WDM system, it’s possible to add some extra channels to it and in that way to increase the total transmission capacity of the used fibre and postpone bottleneck effect of core networks. But using the alternative modulation formats, such as differential phase shift keying with non – return – to zero coding (NRZ – DPSK) or two – states polarization shift keying (2 – POLSK), instead of traditional on – off keying with NRZ coding (NRZ – OOK), it’s possible to achieve the lower bit error ration BER level and channel’s tolerance to transmission impairments, such as inter–channel nonlinearities and crosstalk. The study object of this work is the allowed channel spacing in combined HDWDM systems, if modulation format in each system’s channel are different. For numerical evaluation of modulation formats simultaneous propagation in WDM system, which consists of three channels with 10 Gbit/s bit rate each, simulation model in OptSim software were introduced. The obtained BER values of each channel were compared with the results of traditional systems, which are using only one of mentioned above optical signal modulation formats. These results will allow choosing development strategy in construction of new optical network and improving already existing network infrastructure.