It is well-known that highly-interconnected oscillator networks can synchronize and exhibit a collective mode. Typically, increase of coupling facilitates synchrony. However, in a number of models the increase of interaction strength beyond a certain threshold results in synchrony breaking and appearance of interesting dynamical regimes. We start by consideration of a minimal model of a population of identical oscillators with a nonlinear coupling - a generalization of the popular Kuramoto model. In addition to well-known for the Kuramoto model regimes of full synchrony and full asynchrony, ensembles of nonlinearly coupled oscillators demonstrate nontrivial types of partially synchronized dynamics, in particular the so-called self-organized quasiperiodic states. The analytical treatment of these states is based on the Watanabe-Strogatz ansatz. Theory is illustrated by numerical examples; in particular we demonstrate emergence of similar states in a population of Hindmarsh-Rose neuronal oscillators.
Next, we illustrate our theory by the results of physical experiments. Namely, we analyze collective dynamics of a population of 72 electronic Wien-bridge limit-cycle oscillators with a nonlinear phase-shifting unit in the global feedback loop. With an increase in the coupling strength we first observe formation and then destruction of a synchronous cluster, so that the dependence of the order parameter on the coupling strength is not monotonic. After destruction of the cluster the ensemble remains nevertheless coherent, i.e., it exhibits an oscillatory collective mode. We show that the system is now in a self-organized quasiperiodic state, predicted by our theory. In this state, frequencies of all oscillators are smaller than the frequency of the mean field, so that the oscillators are not locked to the mean field they create and their dynamics is quasiperiodic.
Finally, we demonstrate the emergence of a complex state in a homogeneous ensemble of globally coupled identical oscillators, reminiscent of chimera states in nonlocally coupled oscillator lattices. In this regime some part of the ensemble forms a regularly evolving cluster, while all other units irregularly oscillate and remain asynchronous. We argue that the chimera emerges because of effective bistability, which dynamically appears in the originally monostable system due to internal delayed feedback in individual units. Additionally, we present two examples of chimeras in bistable systems with frequency-dependent phase shift in the global coupling.