One of the benefits of working at MIT is getting a glimpse into the future of technology—from breakthroughs in quantum computing and sustainable energy production to the design of new antibiotics. If you ask me if I have a deep understanding of all these fields, the answer is no. But when researchers invite me to photograph their work for documentation, I am able to understand most of it.
The pleasure of being a scientific photographer is that I must understand what I am documenting to create images that are both informative and credible for the researchers who open their lab doors to me. At their core,
these images themselves exist as a form of experimental data.
Oil droplets containing iron particles on a microscope slide respond to a magnet below. Credit: Felice Frankel
However, with the widespread proliferation of generative artificial intelligence tools, a series of questions urgently need to be explored. Will there come a time when scientists can create 'visualizations' of their research with just a few keystrokes and prompts, viewing these images as records of their work?
Can researchers, journal editors, and readers accurately distinguish artificially synthesized images and understand that they are not records of the real research process?
And finally, a question of personal interest to me:
in the age of AI, is there still a need for roles like scientific photographers to advance science communication?
By personally experimenting with AI image generation tools, I have gained some insights.
Author
Felice Frankel
Scientific Photographer
A research scientist in Chemical Engineering at MIT, she has received numerous awards for the high aesthetic quality of her scientific photographs and her ability to effectively communicate complex scientific information through images. She encourages researchers to question all image manipulation and enhancement techniques. Author of
The visual elements - photography, Picturing science and engineering
, among others.
Truth and Representation
We first need to clarify the fundamental difference between traditional photography and AI-generated images. In the former, each pixel corresponds to a real-world photon, while the latter is constructed through diffusion models. This complex computational process can generate things that appear real but may never have existed.
To explore this difference, with the help of Gaël McGill, a scientific visualization expert at Harvard University, I tried to recreate my classic scientific images using Midjourney and OpenAI's DALL-E.
In 1997, Moungi Bawendi, a chemist at MIT, invited me to photograph his nanocrystals (quantum dots). These crystals fluoresce under UV light, with the emission wavelength varying depending on the crystal size. Bawendi later won the Nobel Prize for this work. He didn't like the first photo I took, where I laid the vials flat on the lab bench and shot from above (see diagram). You can see the air bubbles inside the tubes, indicating how I placed them. This was intentional, and I believe it enhanced the visual appeal of the image.
Three perspectives. The composite image shows three views of vials: the first is from the photographer's perspective with visible air bubbles; the second shows the scientist's focus on color; the third is AI-generated and not a true depiction. Credit: Felice Frankel
The modified second version of the photo was selected for the cover of the November 1997 issue of the
Journal of Physical Chemistry B
. This image not only provided a direct record of the research but also highlighted the importance of collaboration between scientific photographers and researchers, which is an indispensable part of my workflow.
To generate a similar image in DALL-E, I entered the following prompt: "create a photo of Moungi Bawendi’s nanocrystals in vials against a black background, fluorescing at different wavelengths, depending on their size, when excited with UV light."
Images generated by AI
You might think the images generated by the program are aesthetically pleasing, but their authenticity is far from the original photograph. DALL-E generated beaded microparticle structures in the image that were not mentioned in the prompt. This might be because its algorithm, upon retrieving the term "quantum dots" in the underlying model dataset, replaced the original term "nanocrystals".
More concerning is that each vial contains multi-colored microparticle structures, implying that the sample contains a mixture of materials fluorescing at multiple wavelengths, which is contrary to the facts. Furthermore, some particles are depicted scattered on the lab bench surface. Is this treatment based on the model's aesthetic considerations? I think the generated visual effect is very appealing.
Image generated by AI
In my AI generation experiments, the images I obtained were often cartoon-like and difficult to achieve in reality, let alone serve as scientific records. However, technological iteration will eventually break through this barrier. Through in-depth discussions with colleagues in the scientific community and computer science field, we reached a consensus that clear permissive guidelines must be established.
In my view, AI-generated visual content should never be permitted as documented records.
Image generated by AI
The Essential Distinction Between Image Processing and AI Generation
The advent of artificial intelligence means we must clarify three core issues in the field of visual communication: the difference between illustrative diagrams and photographic documentation, the ethical norms of image manipulation, and the urgent need for visual-communication training for scientists and engineers.
Image composition, which is the choice of what to include or exclude, is itself a form of modification of reality. The tools people use are also part of this modification. Every digital camera takes unique photos; Apple iPhones' image algorithms differ significantly from Samsung phones in color enhancement; similarly, the near-infrared images taken by the James Webb Space Telescope, while different from the optical scans of the Hubble Space Telescope, are intended to complement them.
Furthermore, the brilliant colors presented in those stunning cosmic images are all digitally enhanced, creating multi-dimensional interpretations of reality. In this sense,
humans have actually been 'generating images artificially' for many years.
However,
there is a fundamental difference between enhancing photos through software to depict reality and creating virtual reality based on training datasets.
As a scientific photographer, I am well aware of the boundary between illustrative diagrams and documentary images, but I am reserved about whether artificial intelligence programs possess such judgment. Illustrative diagrams or charts subjectively translate and visually describe concepts through symbols, colors, shapes, etc., essentially representing something; documentary images obtained through optical photography or scanning/transmission electron microscopy, though not the physical entity itself, are objective records formed using photons or electrons. The essential difference lies in their purpose.
The core purpose of illustrative diagrams is to describe and clarify research content, an area where generative AI might excel. But for documentary images, the purpose is to restore the real world to the maximum extent possible. Both are essentially acts of modification or artificial generation, which highlights
the necessity for in-depth discussion and the establishment of relevant ethical guidelines before introducing generative AI tools.
Current publishing institutions are equipped with software to detect various forms of image manipulation in existing images (see
Nature
626, 697–698; 2024), but frankly, artificial intelligence programs will eventually be capable of circumventing such protective mechanisms. The academic community is working to build image provenance systems to fully record any modification history of the original image. For example, the forensic photography community, through the global organization "Coalition for Content Provenance and Authenticity," provides technical guidance to camera manufacturers to achieve photo provenance by recording all image processing operations on the device. But unsurprisingly, not all manufacturers have adopted this standard.
The scientific community still has time to build a transparent system and formulate relevant guidelines for AI-generated images.
At a minimum, all generative AI images must be clearly labeled with their attributes, and the creation process and tools used must be clearly explained, including the source image information provided to the AI engine if possible.
However, establishing a provenance list still faces significant challenges.
Two important papers reveal potential privacy and copyright risks in diffusion model applications (N. Carlini et al. Preprint at arXiv https://doi.org/grqmsb (2023); see also go.nature.com/4jqyevn). Copyright attribution only applies to closed systems where training data is known and fully recorded (diffusion models do not yet meet this condition). For example,
Springer Nature
, the publisher of
Nature
journal, recently added a special exception clause to its policy for Google DeepMind's
AlphaFold
program, allowing its model trained on specific scientific datasets to be applied. However, it must be particularly noted that AlphaFold is not a generative AI tool that generates images; its output is structural models (i.e., coordinate data), which subsequently need to be converted into images by researchers (not generative AI tools).
It is reassuring that efforts are underway to address privacy issues. Creators can now use tamper-evident metadata called
Content Credentials
, as described in the Adobe technical guide, which aims to "give creators the credit they deserve and increase transparency in the creative process."
Ethical Standards
For years, I have been advocating for the urgent need for researchers to receive systematic training in the ethics of visual communication, and the widespread application of AI image generation tools further highlights the urgency of related discussions.
For example, I once encountered an engineer who unauthorizedly modified a photo I took for their research and wanted to publish the processed image along with the submitted paper. The researcher did not realize that tampering with images is essentially equivalent to tampering with data. This lack of awareness stems from never having received basic ethical education in image processing and visual communication.
Author's photo and the modified photo
Colleagues in the computer science field point out that while discussions on AI ethics are widespread, they mainly occur outside the scientific community.
What worries me is that the entire research community has not yet fully recognized that image processing is not just an aesthetic issue but an ethical proposition that needs to be taken seriously.
To what extent can an image be modified and still be considered a scientific record? How do we judge whether data is truthfully presented in an image, and whether there are intentional or unintentional omissions?
Facing generative AI visual works that are built from scratch for documentation purposes, entirely based on algorithmic filtering of real-world material, how should their ethical boundaries be defined?
Many questions remain to be answered.
Future Vision
It is clear that generative AI images will be part of our future. Although most may fall into the category of illustrative diagrams, we must squarely face their potential use as scientific records. Based on this, the research community urgently needs to establish guiding principles, requiring researchers submitting papers containing images to answer at least the following questions:
1. Is the image generated by AI? If yes, it must be clearly labeled and include metadata identifying its attributes.
2. What specific generative AI model and version were used?
3. What prompts were used to generate the image?
4. Was an image used to assist the prompt? If yes, submit the image as well and provide source information.
The Role of the Photographer
To answer the question posed at the beginning of this article—is there still a place for scientific photographers in the age of AI? I sought an answer from OpenAI's ChatGPT. Here is a condensed version of its reply:
"In the realm of AI-generated images, photographers documenting scientific objects play a unique role, providing expertise, authenticity, and a critical perspective in a field where accuracy and representation of reality are crucial."