The Photonics Spotlight > 2025-04-01

Have AI Bots Like Chat-GPT Got Better on Photonics Questions?

Previous Tests

In the Photonics Spotlight article of 2023-09-06, I investigated with a few examples how ChatGPT and Google Bard (predecessor of Gemini) performed on photonics questions. While they already did remarkably well on language transformations and coding, the results with photonics questions were quite mixed. For example, I got terrible results from Google Bard concerning the working principles of fiber amplifiers, and similar kinds of nonsense also from ChatGPT concerning the concept of intracavity pumping of lasers. It became very clear than one could in no way trust their claims.

About 1.5 years later, one could imagine that the progress on AI, based on huge investments by various companies, resulted in substantial improvements, perhaps also increased reliability concerning photonics questions. So I now investigated this a bit.

Some Test Cases

Laguerre–Gaussian Beams

I recently got a question from an Encyclopedia user concerning the Rayleigh length of Laguerre–Gaussian beams: “What is the Rayleigh length for different Laguerre–Gaussian modes?” When I now asked ChatGPT 4o exactly that question, it first correctly explained the Rayleigh length or a Gaussian beam, then gave some correct explanations on Laguerre–Gaussian modes including a formula for the increased beam waist radius (but without explaining on what definition that is based). Thereafter, it claimed that an effective Rayleigh length can be defined as ($z_\textrm{R, eff} = z_\textrm{R} \: (2p + |l| + 1)$) – which for higher-order modes is obviously larger than the value for a Gaussian beam. It added the strange sentence: “This is not the physical Rayleigh length from the paraxial equation, but it's useful in practice to describe the divergence and focusing behavior of higher-order modes.” Well, that's indeed quite wrong and misleading.

I then tried with ChatGPT 4.5 (offered as research preview). Here, the answer was fairly clear and essentially correct; only, it did not explain what exactly you should enter for ($w_0$) in the given formula. I asked that as a follow-up question, and it correctly and clearly answered: “Although LG modes have ring-shaped intensity patterns, the ($w_0$)​ used in the Rayleigh length calculation still refers to the fundamental Gaussian beam waist radius defining the overall Gaussian envelope.”

I also asked the original question to Google Gemini 2 flash. It produced a quite verbose output, mentioning various irrelevant details and coming to the conclusion: “In summary, while the basic Rayleigh length formula applies, the specific p and l indices of LG modes and the M² factor influence the effective Rayleigh length.” So in the end you still don't know.

For the correct answer, see my Encyclopedia article on Rayleigh length.

Working Principle of Fiber Amplifiers

Like in my previous test, I entered: “Explain the working principles of fiber amplifiers.”

Gemini 2.0 Flash now gave a good answer – maybe a bit verbose, but pretty clear and helpful. This is strong progress after the very poor results of my previous test.

ChatGPT 4.5 (research preview) gave a largely useful answer, although with some deficiencies:

• It claimed a population inversion, although in reality things are somewhat more tricky in the context of quasi-three-level laser gain media. (At longer signal wavelengths, we can have gain with well under 50% in the excited Stark level manifold).
• It said: “When a weak signal photon (at 1550 nm for EDFA) enters the fiber, …” – are there weak and strong photons??
• It claimed a low noise figure particularly for erbium-doped fiber amplifiers, although that is not generally true, particularly not for short signal wavelengths.

To my surprise, ChatGPT 4o was actually a bit better than ChatGPT 4.5 on this question. Presenting a brief answer, it avoided various additional problematic statements.

Alignment Sensitivity of an Optical Resonator

The question: “Is the alignment sensitivity of an optical resonator generally higher near the edges of the stability zones?”

ChatGPT 4.5 (the largest tested model) claimed: “Yes, the alignment sensitivity of an optical resonator generally increases as the cavity parameters approach the edges of the stability zones.” It gave some reasons which sounded plausible, but it collided with my knowledge, which lead me to do a follow-up question: “Consider alignment sensitivity in terms of mode axis position changes resulting from small angular changes of mirror alignment.” The obtained answer: “Considering alignment sensitivity specifically in terms of mode-axis displacement caused by small angular changes in mirror alignment, the sensitivity is indeed higher near the edges of the resonator stability zones.” Now that is definitely not true. In reality, the alignment sensitivity in that (useful) sense diverges at one of the four stability edges of a linear resonator (having two stability zones), but not at the three other stability edges. It can even be the lowest possible at a stability edge:

Of course, I would still not conclude that it would be wise to operate a resonator there: a diverging beam radius on at least one end mirror is certainly not good to have, and also you could then easily drift into the unstable region.

I did the same test with Gemini Deep Research. It took quite long, trying to use reputable sources (with my article popping up several times), and finally presented a rather long research report. That tends to the opinion that alignment sensitivity is particularly high at all stability edges, but does not present convincing arguments and does not seem to be fully sure about its position. Well, that's at least better than claiming wrong things with full confidence.

I must admit that this is a difficult problem, where one should not expect a great answer from a machine.

Intracavity Pumping of Lasers

Having seen some questionable answers related to intracavity pumping of lasers, I asked: “Could a Ti:sapphire laser be made more efficient by intracavity pumping?”

ChatGPT 4.5 confidently answered: “Yes, intracavity pumping can significantly enhance the efficiency of a Ti:sapphire laser, although the improvements depend strongly on how carefully the intracavity arrangement is implemented.” I was curious to analyze the given reasons:

• It first explained potential advantages of intracavity pumping, such as increasing pump absorption efficiency and pump intensities.
• From that, it concluded things like: “Intracavity-pumped Ti:sapphire lasers exhibit lower threshold pump powers, enabling compact, more efficient laser setups.” Also: “It is common for intracavity-pumped Ti:sapphire lasers to achieve higher slope efficiencies (15–30% or better) compared to externally pumped setups (~10–15% typical).” But in reality, I never heard that anybody is making such lasers!
• How could such a laser be made? ChatGPT said: “Intracavity pumping involves placing the Ti:sapphire crystal inside the cavity of the pump laser (often a diode-pumped solid-state laser, like Nd:YVO₄ or Nd:YAG).” But how can laser light from those be absorbed by Ti:sapphire? Later on, it considered spectral matching and found that intracavity frequency doubling is required. There are indeed many Nd:YAG lasers and similar with intracavity frequency doubling, but the frequency-doubled light is normally not resonant: So even if you put your Ti:sapphire crystal into that laser resonator, this won't be intracavity pumping – and indeed it makes no sense to put the crystal there; you put it outside the pump laser resonator.

In conclusion, if you believe all this, you will be quite mislead concerning how such a laser could reasonably be built. In reality, I don't think we will anytime soon see any intracavity-pumped Ti:sapphire laser, since there is hardly a suitable type of pump laser for that, and also because pump absorption is not a major concern of existing Ti:sapphire lasers. So there is not really a reason to go that direction.

Time Zones and File Dates

I also recently checked a non-photonics question: essentially, how daylight saving time is applied to the modified dates of Windows files. ChatGPT generally performs quite well on such IT questions, but here it also created a big mess. While it correctly explained that the application of DST offset of one hour depends on whether the viewing time (and not the last modified data) was in daylight saving time, it subsequently delivered code which worked on other assumptions. When told to be wrong, it offered corrections which were again wrong. There still seem to be some problems with relatively basic logic – not only missing special knowledge like in photonics.

Conclusions

So Far, No Clear Progress

Although these examples can, of course, not be regarded as completely representative, we can conclude quite clearly that expecting reliable answers to photonics questions – even not extremely difficult ones – is still not realistic at all. I actually do not see a substantial overall improvement compared with the previous test 1.5 years ago, even when using some of the latest research-type language models.

I am actually not surprised. Despite all the technical progress made in recent years, an essential limitation remains: there is a limited amount of training material for such a specialized topic as photonics. And it is a fact that in the available texts there are many statements which are not fully clear and in many cases also not fully correct. Naturally, a language model can only find out what one would typically say on certain things, and we cannot expect it to critically (and reliably) deviate from that based on thorough reasoning. Therefore, these language models work well only on topics where fully correct statements are abundant and not diluted with misleading and wrong stuff.

Read the Reliable Sources!

The clear consequence of that insight is that especially in specialized topics like photonics, you cannot possibly rely on the results of such language models without thoroughly checking yourself all essential issues. If you need reliable results and do not want or cannot do that thorough checking, the only choice is to use a very trustworthy source, which for now and in the foreseeable future needs to be human expertise. This is what RP Photonics offers, for example.

AI tools also increasingly use resources like ours; we have actually started monitoring that a while ago and see very substantial amounts of our webpages retrieved by these AI bots. ChatGPT is the leading one; it has retrieved around 66,000 of our pages in last March. But usage of that content does not guarantee correct statements: it also happens that a source with fully correct content is used as a reference for a false AI-created statement! Keep in mind: If you want to be sure that RP Photonics said it, you need to read the RP Photonics texts yourself!

Reliable AI Tools

Note that RP Photonics offers its own AI-based tools – made such that the results are far more reliable than what you would get from ChatGPT, for example. Our solution is that we begin with a semantic search based on our vast amount of high-quality content, which we then use as a thorough basis for mild LLM processing. We encourage our users to read the identified article sections rather than trying to replace these with LLM-generated text.

We have considered in detail a substantially more heavy use of AI: implementing a photonics chat bot based on a language model which has been fine-tuned with all our content. However, we found that this is for the foreseeable future not the right approach. A core reason is that we could not safely avoid that the language model uses information which it picked up somewhere else (in the general training of the original model). Even where it would fully rely on our content, it could still produce wrong conclusions. Therefore, we decided to essentially work with semantic search, which is often very helpful and reliable at the same time.

This article is a posting of the Photonics Spotlight, authored by Dr. Rüdiger Paschotta. You may link to this page and cite it, because its location is permanent. See also the RP Photonics Encyclopedia.

Note that you can also receive the articles in the form of a newsletter or with an RSS feed.

Questions and Comments from Users