For multivariate linear regression systems,using the coupling identification concept and the multi-innovation identification theory,this paper discusses a multivariate stochastic gradient algorithm,a multivariate multi-innovation stochastic gradient algorithm,and an interval-varying multivariate multi-innovation gradient algorithm,decomposes a multivariate system into several subsystems,and presents a coupled subsystem stochastic gradient algorithm,a coupled stochastic gradient algorithm,a coupled subsystems multi-innovation stochastic gradient algorithm and a coupled multi-innovation stochastic gradient algorithm.These methods are extended to multivariate pseudo-linear moving average systems and multivariate pseudo-linear autoregressive moving average systems.Finally,this paper gives the steps and diagrams for computing the parameter estimates using several typical coupled stochastic gradient algorithms and coupled multi-innovation stochastic gradient algorithms.