To identify non-periodic neural rhythm to be chaos or stochasticity has been an important scientific thesis. A kind of non-periodic spontaneous firing pattern, whose behavior is transition between period-k burst in a string and period-k+1 burst in a string (k=1,2), lying between period-k bursting pattern and period-k+1 bursting pattern, is found in the experimental neural pacemaker. The deterministic structures of the firing are identified by nonlinear prediction and first return map of the interspike intervals (ISIs) series. The co-existence of the period-k bursting and period-k+1 bursting is manifested in the deterministic theoretical neuronal model, Chay model. Non-periodic firing patterns similar to the experimental observation are simulated in the co-existing parameter region, implying that the firing pattern is transition between two kinds of bursts induced by noise. A binary series can be acquired by transforming two kinds of bursts to symbols 0 and 1, respectively. The stochastic dynamics within the transitions between two kinds of bursts are detected by probability analysis on the binary series. It not only shows that the rhythm is stochastic firing with deterministic structures instead of chaos, but also provides the typical examples and effective methods to intensively identify the chaotic and stochastic firing patterns in a real nervous system.