Case Study: Transverse Mode Instability in a Kilowatt Fiber Amplifier
Key questions:
- Up to which output power can a given ytterbium-doped fiber amplifier be operated before transverse mode instability sets in?
- In which part of the fiber does the instability build up?
- Which design measures raise the threshold, and which surprisingly don't?
- How reliable is such a threshold estimate?
Ytterbium-doped high-power fiber amplifiers can deliver kilowatts of output power with nearly diffraction-limited beam quality, and are widely used, e.g. for laser material processing. Since about 2010, however, it has become clear that one of the main limitations of their power scaling – besides nonlinear effects such as stimulated Raman scattering – is transverse mode instability (TMI): Above a certain threshold power, the beam quality suddenly degrades. When designing such an amplifier, one should therefore know in advance how close it operates to that threshold and which design parameters can raise it. Finding that out experimentally is tedious and expensive, as each design variant requires a fiber and a high-power test setup. In this case study, we show how a numerical model can give such answers within seconds, and what can be learned from it.
The Amplifier
We consider a cladding-pumped fiber amplifier based on an ytterbium-doped large mode area fiber with a step-index core of 25 µm diameter and a numerical aperture of 0.06, a pump cladding of 400 µm diameter and a length of 4 m. The core is uniformly Yb-doped. A signal with 10 W at 1064 nm is amplified with 1 kW of pump power at 976 nm, launched in the forward direction (co-pumping). Amplified spontaneous emission (ASE) is taken into account, but turns out to be weak.
At 1064 nm, the fiber core (with low numerical aperture of 0.06) supports not only the fundamental mode LP01, but also the LP11 mode. The amplifier is operated with the signal in LP01, and we want to keep it there. At high powers, however, TMI can transfer power to LP11, which degrades the beam quality and makes the output beam profile fluctuate.
For the simulations, we use the software RP Fiber Power. It offers a TMI threshold estimate based on a linear stability analysis of the steady state of the amplifier, which takes only a few seconds. (The same estimate is available in the Power Forms for continuous-wave fiber amplifiers and for fiber lasers with amplifiers, there for the amplifier stages.) For comparison, a full-blown simulation of TMI, telling us what happens in the regime of strong TMI, could take hours for a single configuration.
The signal output power for 1 kW pump is 794 W. The dissipated power is 71 W, with a maximum heat load of 28 W/m near the pump input. Most of the heat arises from the quantum defect, i.e., the energy difference between pump and signal photons.
How TMI Arises
The mechanism is explained in detail in the encyclopedia article on transverse mode instability (see also [1, 5]); here is a short summary:
- A weak LP11 wave interferes with the strong LP01 wave. Since the two modes have different propagation constants, their beat pattern is an intensity grating along the fiber, with a period of the beat length – here 3 mm.
- Through the heat generation in the doped core and the thermo-optic effect, that intensity grating produces a refractive index grating, which can scatter light from one mode into the other.
- A net power transfer requires a phase shift between the index grating and the intensity pattern. That cannot happen for a stationary grating. However, it occurs if the LP11 wave has a slightly lower optical frequency (offset in the kHz range): The intensity pattern then moves slowly along the fiber, and the temperature grating lags behind, because heat conduction needs some time. That delay creates a phase shift which enables energy transfer to the LP11 mode. At high enough pump power, this leads to exponential growth of LP11 power such that the beam quality is spoiled, and operation becomes unstable.
The following diagram shows that index grating for our amplifier, as an xz cut through the fiber at the position where the coupling is strongest:
One can see that the index maxima indeed lag behind the intensity maxima by about 0.3 mm – roughly a tenth of the beat length. That small shift is the key to the instability, as explained above. Note also that the temperature grating extends somewhat into the cladding, and that its fronts are curved: The farther from the fiber axis, the more the temperature maxima lag behind. This is because the heat needs time to diffuse outwards; at the edge of the displayed area, about 6 µm outside the core, the lag is already about a quarter of the beat length.
Threshold and Frequency
The LP11 wave grows exponentially along the fiber, with an exponent ($G$) which depends on the frequency offset. The question is from which initial level it grows. Even with a perfect launch and no technical noise, quantum noise provides a minimum LP11 seed, which can be simulated with a seed power of the order of 10−16 W within the relevant bandwidth. If we define the threshold as the point where the LP11 content at the output reaches 1 %, the exponent required for 10 W input power is
$$G_\mathrm{th} = \ln \left( \frac{0.01 / 0.99}{10^{-16} \: \mathrm{W} / 10 \: \mathrm{W}} \right) \approx 34.5$$The following diagram shows the exponent versus the frequency offset:
For 1 kW pump power, the maximum exponent is 25.7, well below 34.5. By increasing the pump power, we find the threshold at 1.32 kW pump power, where the signal output power is 1.04 kW. The most strongly growing frequency offset is then 3.6 kHz. That frequency is essentially determined by the thermal diffusion time across the region of the mode – a fraction of a millisecond for the distance between the lobes of LP11.
So, if only quantum noise seeds the instability, we can operate this amplifier at 1 kW pump power with some safety margin, but not much more than about 1 kW of output power can be expected. Note, however, that this margin corresponds to a factor of only about 7000 in seed power: An LP11 seed of about 10−12 W near the TMI frequency would suffice to reach the threshold already at 1 kW pump power. Technical noise can provide such seeds (see below).
Where the Instability Builds Up
One might expect that TMI arises mainly where the heat load is highest, or near the output where the signal power is highest. Neither is quite correct:
The local TMI gain coefficient has its maximum of about 20 m−1 only 0.4 m from the input. Near the input, it is small because the coupling is proportional to the signal power, which is only 10 W there. Further downstream, the signal power is high, but the TMI gain decreases again. One reason is that the heat load decreases with the pump power. More important, however, is gain saturation: What matters for TMI is not the heat load itself, but how much the heat generation changes when the local signal intensity is modulated by the beat pattern. Under strong saturation, a higher signal intensity hardly increases the extracted power, so that the heat modulation becomes weak [2].
The consequence for the LP11 power is shown in the next diagram:
At the threshold, the LP11 power grows from 10−16 W to about 10 W at the output – by 17 orders of magnitude, with 13 of those within the first 1.5 m. The remaining 2.5 m, where the signal power is highest, contribute comparatively little.
Measures Raising the Threshold
A major benefit of modeling and simulation is that we can study the dependence on various parameters, which aids device optimization and is far faster and cheaper than exploring all this experimentally. As our threshold estimate takes only a few seconds, we can easily explore how the TMI threshold depends on various parameters.
Signal Input Power
A higher signal input power (from a more powerful preamplifier stage) raises the TMI threshold: from 0.92 kW for 1 W input to 1.42 kW for 100 W input. There are two reasons:
- A stronger input signal saturates the gain more strongly from the beginning, which weakens the heat modulation in the input region, where the instability builds up.
- The relative LP11 seed level is lower, so that a somewhat larger exponent is required.
The possible improvement is significant, although not dramatic:
Pump Direction
With counter-pumping (pump launched at the signal output end), the threshold rises from 1.04 kW to 1.22 kW. Here, the input part of the fiber, where the signal is still weak, is only weakly pumped and generates little heat; most heat is generated near the output end, where the gain is strongly saturated by the high signal power. Interestingly, the highest threshold (1.26 kW) is obtained not for pure counter-pumping, but for 70 % to 80 % of the pump power launched in backward direction. With pure counter-pumping, the heat is concentrated at the output end, where the signal power – and thus the coupling – is highest.
Note that counter-pumping substantially increases the maximum heat load per unit length (at the threshold 67 W/m instead of 37 W/m), which may require more efficient cooling. In the end, one might decide for bidirectional pumping, e.g. with equal powers from both sides.
Higher-order Mode Losses
Additional losses of LP11, as can be introduced by coiling the fiber (bend losses), raise the threshold – but less than one might expect: 10 dB/m of extra loss raises it by only 25 %. The reason is that the TMI gain coefficient in the input region is far higher (up to 20 m−1) than such a loss (2.3 m−1).
Another interesting direction is optimizing the fiber design for increased LP11 losses.
Avoiding Background Absorption
Here comes a less pleasant surprise: Background absorption of the signal light, as can be caused by photodarkening, lowers the threshold drastically. Already 0.01 dB/m reduces it from 1.04 kW to 0.84 kW, and 0.1 dB/m to 0.28 kW. The absorbed power only moderately adds heat, but this heat follows any modulation of the signal intensity fully, while the modulation of the quantum defect heating is strongly reduced by gain saturation. Therefore, fibers with low photodarkening are essential for high TMI thresholds [4].
Pump Wavelength
Finally, we compare different pump wavelengths. Since the pump absorption per unit length depends strongly on the pump wavelength, we adjust the fiber length such that the small-signal pump absorption remains the same (10.5 dB):
| Pump wavelength | Fiber length | Threshold, co-pumped | Threshold, counter-pumped |
|---|---|---|---|
| 915 nm | 12.6 m | 1.82 kW | 3.41 kW |
| 940 nm | 28.3 m | 3.79 kW | 8.21 kW |
| 976 nm | 4.0 m | 1.04 kW | 1.22 kW |
| 981 nm | 17.0 m | 2.51 kW | 4.09 kW |
Although the quantum defect is larger for 915 nm and 940 nm, pumping away from the absorption peak at 976 nm raises the threshold substantially. This is in line with experimental findings, where the threshold increased by a factor of about 1.8 when changing the pump wavelength from 976 nm to 915 nm [3] and by a factor of about 2.2 when changing to 981 nm (as reported in [7]); our model gives factors of 1.75 and 2.41 for these cases.
A longer fiber is also good in terms of heat generation density, i.e., one can use less aggressive cooling. On the other hand, it increases nonlinear optical effects such as stimulated Raman scattering and Brillouin scattering, which also need to be considered for device designs.
What Does Not Help
One might think that distributing the heat over a longer fiber helps – for example, using a fiber with half the doping concentration and double length, so that the pump absorption remains the same. The maximum heat load per unit length is then halved. However, the model gives exactly the same threshold.
So a lower heat load per unit length is no lever by itself – at least as long as there are no concentration-dependent processes (e.g. quenching) and no background absorption, which would favor shorter fibers.
How Reliable Is the Estimate?
Absolute threshold values from such models should be taken as estimates. Comparisons with published model calculations show deviations of up to about 20 %, and the inputs are often not well known:
- Seed level: The relevant seed is the part of the LP11 field which fluctuates relative to LP01 in the kHz range; quantum noise is only a lower limit. Fortunately, the seed level enters only logarithmically: a tenfold seed power lowers the threshold by about 6 %. Still, a seed of 10−8 W instead of 10−16 W would halve the threshold. Such strong seeds can occur in practice: Intensity noise of the pump in the kHz range modulates the heat load and thus the phase relation of the modes, and acting on a static LP11 content from an imperfect launch, it creates fluctuating LP11 light at the relevant frequencies. In experiments, a few percent of pump intensity noise lowered the threshold by more than half [6]. Low-noise pump drivers and a clean launch are therefore important. Note that only noise components near the TMI frequency (here a few kHz) matter, not the total intensity noise. As the seed power scales with the square of the noise amplitude, each factor of 10 in that noise amplitude changes the threshold by roughly 13 %.
- Index profile: The refractive index profiles of commercially available fibers are often not known to the users. However, simulations show that typical deviations from the step-index profile, such as a narrow central index dip or ripples, change the threshold by less than 10 %.
- Higher-order mode losses and photodarkening: As shown above, these can have substantial effects but are often not precisely known. Photodarkening implies losses rising with time – and the strength of that can strongly depend on operation parameters, such as the strength of Yb excitation. Note that such parameters can also be monitored and optimized with modeling.
- Signal linewidth: The model assumes that the signal is coherent over the walk-off between the two modes. For this fiber, that holds for linewidths below about 200 GHz – far more than the few GHz which are typically used for suppressing Brillouin scattering. For very high signal linewidths, TMI can be reduced by washing out the interference pattern.
Generally, trends, as found when comparing design options, are more reliable than absolute values. For example, the dependence on the pump wavelength agrees well with experimental findings, as shown above. In practice, a measured threshold of an existing amplifier can be used to calibrate uncertain parameters, and the model then predicts the effect of design changes.
Conclusions
The RP Fiber Power software is an invaluable tool for such work – very powerful and at the same time pretty easy to use!
You can learn various things from this study:
- The TMI threshold of a kilowatt fiber amplifier can be estimated within seconds; the result is an estimate, but well suited for comparing design options.
- The instability builds up mainly in a relatively short part of the fiber, near the input, where the signal is already quite strong but the gain is not yet strongly saturated.
- Gain saturation is the key to understanding many dependencies: higher signal input power, (partial) counter-pumping and pumping away from the absorption peak all raise the threshold by strengthening the saturation.
- Background absorption, e.g. from photodarkening, can lower the threshold drastically, as its heat is not reduced by saturation.
- Spreading the heat over a longer fiber with lower doping, keeping the pump absorption constant, does not help.
- The calculated threshold assumes seeding by quantum noise. Intensity noise of the pump in the kHz range, together with a small static LP11 content from the launch, can lower it substantially.
Bibliography
| [1] | A. V. Smith and J. J. Smith, “Mode instability in high power fiber amplifiers”, Opt. Express 19 (11), 10180 (2011); doi:10.1364/OE.19.010180 |
| [2] | A. V. Smith and J. J. Smith, “Increasing mode instability thresholds of fiber amplifiers by gain saturation”, Opt. Express 21 (13), 15168 (2013); doi:10.1364/OE.21.015168 |
| [3] | R. Tao et al., “Mitigating of modal instabilities in linearly-polarized fiber amplifiers by shifting pump wavelength”, J. Opt. 17 (4), 045504 (2015); doi:10.1088/2040-8978/17/4/045504 |
| [4] | H.-J. Otto et al., “Impact of photodarkening on the mode instability threshold”, Opt. Express 23 (12), 15265 (2015); doi:10.1364/OE.23.015265 |
| [5] | C. Jauregui, C. Stihler and J. Limpert, “Transverse mode instability” (review paper), Adv. Opt. Photon. 12 (2), 429 (2020); doi:10.1364/AOP.385184 |
| [6] | C. Stihler et al., “Intensity noise as a driver for transverse mode instability in fiber amplifiers”, PhotoniX 1, 8 (2020); doi:10.1186/s43074-020-00008-8 |
| [7] | M. N. Zervas, “Transverse mode instabilities in high power fiber amplifiers: a linear stability analysis”, Opt. Express 34 (14), 25172 (2026); doi:10.1364/OE.601786 |
