Generated by All in One SEO v5.0.0.1, this is an llms.txt file, used by LLMs to index the site.

# Jan Zahálka

AI & Security

## Sitemaps

- [XML Sitemap](https://zahalka.net/sitemap.xml): Contains all public & indexable URLs for this website.

## Posts

- [zahalka.net: AI & security blog](https://zahalka.net/ai_security_blog/) - Welcome to my blog! I cover AI & security. The core topic is security of AI models: defending AI models from malicious attackers that attempt to disrupt the model or manipulate it into unintended behaviour. Occasionally, I may write about general safety, robustness, trustworthiness, or transparency of AI, or about developments in general cybersecurity.
- [How to intuitively understand adversarial attacks on AI models](https://zahalka.net/ai_security_blog/2023/06/how-to-intuitively-understand-adversarial-attacks-on-ai-models/) - How is it possible that we can make anything look like something completely different in the eyes of an AI model? This post brings a real-world-inspired intuition of adversarial attacks on AI models.
- [What's more powerful than one adversarial attack?](https://zahalka.net/ai_security_blog/2024/02/whats-more-powerful-than-one-adversarial-attack/) - Using a single attack won't do, unless you are in a Hollywood film. This post covers AutoAttack, the pioneer ensemble adversarial attack, and shows how to test the adversarial robustness of AI models more rigorously.
- [Can ChatGPT read who you are?](https://zahalka.net/ai_security_blog/2024/01/can-chatgpt-read-who-you-are/) - ChatGPT is excellent in extracting structured information from text. Can it evaluate our personality traits? This post describes our work on LLM personality assessment, accepted to the CAIHu workshop @ AAAI '24.
- [Elves explain how to understand adversarial attacks](https://zahalka.net/ai_security_blog/2024/01/elves-explain-how-to-understand-adversarial-attacks/) - Intuitive understanding of adversarial attacks is core for understanding AI security. This post aims to explain adversarial attacks with... Elves (instead of technical terminology).
- [A cyberattacker's little helper: Jailbreaking LLM security](https://zahalka.net/ai_security_blog/2023/12/a-cyberattackers-little-helper-jailbreaking-llm-security/) - Attacks, lies, and deceit to bypass the security of (an older version of) ChatGPT. Jailbreaking is an open LLM security challenge, as LLM services should not assist in malicious activity.
- [Judging LLM security: How to make sure large language models are helping us?](https://zahalka.net/ai_security_blog/2023/11/judging-llm-security-how-to-make-sure-large-language-models-are-helping-us/) - Large language models (LLMs) have taken the world by storm, but LLM security is still in its infancy. Read about our contribution: a comprehensive, practical LLM security taxonomy.
- [AI security @ CVPR '23](https://zahalka.net/ai_security_blog/2023/06/ai-security-cvpr-23-introduction-stats/) - CVPR '23 has brought a large number of new, exciting AI security papers. This post kicks off a blog post series covering the work with an introduction, paper stats, and overall topical structure.
- [AI security @ CVPR '23: Honza's highlights & conclusion](https://zahalka.net/ai_security_blog/2023/11/ai-security-cvpr-23-honzas-highlights-conclusion/) - This post presents "Honza's highlights"—CVPR '23 AI security papers that are worthy of your attention and have not received the official highlight status—and conclusions from CVPR '23.
- [Reality can be lying: Deepfakes and image manipulation @ CVPR '23](https://zahalka.net/ai_security_blog/2023/10/reality-can-be-lying-deepfakes-and-image-manipulation-cvpr-23/) - Deepfakes & image manipulation are increasingly used for spreading fake news or falsely incriminating people, presenting a security and privacy threat. This post summarizes CVPR '23 work on the topic.
- [Privacy attacks @ CVPR '23: How to steal models and data](https://zahalka.net/ai_security_blog/2023/10/privacy-attacks-cvpr-23-how-to-steal-models-and-data/) - This post summarizes CVPR '23 work on privacy attacks that threaten to steal an AI model (model stealing) or its training data (model inversion).
- [Cheatsheet of AI security papers from CVPR '22](https://zahalka.net/ai_security_blog/2023/06/cheatsheet-of-ai-security-papers-from-cvpr-22/) - All AI security papers from CVPR '22 with paper link, categorized by attack type.
- [Backdoor attacks & defense @ CVPR '23: How to build and burn Trojan horses](https://zahalka.net/ai_security_blog/2023/09/backdoor-attacks-defense-cvpr-23-how-to-build-and-burn-trojan-horses/) - Backdoor (or Trojan) attacks poison an AI model during training, essentially giving attackers the keys. This post summarizes CVPR '23 research backdoor attacks and defense.
- [From "maybe" to "absolutely sure": Certifiable security at CVPR '23](https://zahalka.net/ai_security_blog/2023/09/from-maybe-to-absolutely-sure-certifiable-security-at-cvpr-23/) - Certifiable security (CS) gives security guarantees to AI models, which is highly desirable for practical AI applications. Learn about CS work at CVPR '23 in this post.
- [AI security @ CVPR '22: Non-classic adversarial attacks research](https://zahalka.net/ai_security_blog/2023/05/ai-security-cvpr-22-non-classic-adversarial-attacks-research/) - One of the most interesting aspects of cybersecurity is the diversity of attack vectors, or paths through which an attack can succeed. How does AI security fare in this regard? This post focuses on the diversity of attacks on computer vision models.
- [How to see properly: Adversarial defense by data inspection](https://zahalka.net/ai_security_blog/2023/09/how-to-see-properly-adversarial-defense-by-data-inspection/) - Data inspection is a promising adversarial defense technique. Inspecting the data properly can reveal and even remove adversarial attacks. This post summarizes data inspection work from CVPR '23.
- [AI security @ CVPR '22: Model inversion attacks research](https://zahalka.net/ai_security_blog/2023/06/ai-security-cvpr-22-model-inversion-attacks-research/) - One of the key AI security tasks is protecting data privacy. A model inversion attack can steal training data directly from a trained model, which should be prevented. This post covers CVPR '22 work on model inversion attacks.
- [Adversarial training: a security workout for AI models](https://zahalka.net/ai_security_blog/2023/08/adversarial-training-a-security-workout-for-ai-models/) - Adversarial training (AT) amends the training data of an AI model to make it more robust. How does AT fare against modern attacks? This post covers AT work presented at CVPR '23.
- [Better model architecture, better adversarial defense](https://zahalka.net/ai_security_blog/2023/08/better-model-architecture-better-adversarial-defense/) - Adversarial defense is a crucial topic: many attacks exist, and their numbers are surging. This post covers CVPR '23 work on bolstering model architectures.
- [Which model architecture is the best in adversarial defense?](https://zahalka.net/ai_security_blog/2023/08/which-model-architecture-is-the-best-in-adversarial-defense/) - A mini-tool for comparison of adversarial defense of various computer vision model architectures, based on the CVPR '23 work by A. Liu et al.
- [Real-world AI security: Physical adversarial attacks research from CVPR '23](https://zahalka.net/ai_security_blog/2023/08/real-world-ai-security-physical-adversarial-attacks-research-from-cvpr-23/) - Physical adversarial attacks fool AI models with physical object modifications, harming real-world AI security. This post covers CVPR '23 work on the topic.
- [From AI security to the real world: Physical adversarial attacks research @ CVPR '22](https://zahalka.net/ai_security_blog/2023/05/from-ai-security-to-the-real-world-physical-adversarial-attacks-research-cvpr-22/) - This post is a part of the AI security at CVPR '22 series. Physical adversarial attacks At a glance, attacks on computer vision models might not sound so serious. Sure, somebody's visual data processing pipeline might be compromised, and if that's a criminal matter, let the courts decide. Big deal, it doesn't really impact me that
- [From one model to another: Transferable attacks research @ CVPR '23](https://zahalka.net/ai_security_blog/2023/07/from-one-model-to-another-transferable-attacks-research-cvpr-23/) - This post summarizes the CVPR '23 work on transferable attacks, optimized on a surrogate model controlled by the attacker to also work on black-box targets.
- [New adversarial attacks on computer vision from CVPR '23](https://zahalka.net/ai_security_blog/2023/07/new-adversarial-attacks-on-computer-vision-from-cvpr-23/) - Adversarial attacks are a core discipline of AI security. This post summarizes pioneering adversarial attacks on computer vision models seen at CVPR '23 that focus on underexplored tasks of computer vision or bring a new view on attack methodology.
- [The best AI security papers from CVPR '23: Official highlights](https://zahalka.net/ai_security_blog/2023/07/the-best-ai-security-papers-from-cvpr-23-official-highlights/) - The AI security papers from CVPR '23 among the top-ranked papers by reviewer score.
- [AI security @ CVPR '22: Image manipulation & deepfake detection research](https://zahalka.net/ai_security_blog/2023/06/ai-security-cvpr-22-image-manipulation-deepfake-detection-research/) - Image manipulation is an attack that alters images to change their meaning, create false narratives, or forge evidence. This post summarizes AI security work on this topic presented at CVPR '22.
- [AI security @ CVPR '22: Backdoor/Trojan attacks research](https://zahalka.net/ai_security_blog/2023/05/ai-security-cvpr-22-backdoor-trojan-attacks-research/) - A backdoor or Trojan attack compromises the model to produce outputs desired by the attacker when an input is manipulated in a certain way. Backdoor attacks give attackers a permanent unauthorized security pass, which poses a great AI security risk. In this post, I cover backdoor attacks research at CVPR '22.
- [Are there guarantees in AI security? Certifiable defense research @ CVPR '22](https://zahalka.net/ai_security_blog/2023/05/are-there-guarantees-in-ai-security-certifiable-defense-research-cvpr-22/) - This post is a part of the AI security at CVPR ’22 series. The issue of CV and AI security can feel quite scary. Stronger and more sophisticated attacks keep coming. Defense efforts are a race that must be run, but cannot be definitively won. We patch the holes in our model, then better attacks
- [AI & security @ CVPR '22: Classic adversarial attacks research](https://zahalka.net/ai_security_blog/2023/05/ai-security-cvpr-22-adversarial-attacks-defense-research-part-1/) - This blog post is a part of the AI & security at CVPR '22 series. Here I cover the adversarial attack terminology and research on classic adversarial attacks. Terminology and state of the art The cornerstone of AI & security research, and indeed the classic CV attack, is the adversarial attack, first presented in the
- [How secure is computer vision? AI & security at CVPR '22](https://zahalka.net/ai_security_blog/2023/05/how-secure-is-computer-vision-ai-security-at-cvpr-22/) - Computer vision (CV) is one of the vanguards of AI, and its importance is rapidly surging. For example, CV models perform personal identity verification, assist physicians in diagnosis, enable self-driving vehicles. There has been a remarkable increase in performance of CV models, dispelling much of the doubt related to their efficiency. With their increased involvement

## Pages

- [Welcome!](https://zahalka.net/) - Hi, I am Jan, and I work on AI & security. The performance, impact, and importance of AI is rapidly increasing, and my mission is to make it more secure, trustworthy, respectful of privacy, and robust. Presently, I am a researcher at the Czech Technical University in Prague (CTU) and the founder & AI specialist
- [Publications](https://zahalka.net/publications/) - For the most up-to-date list of my publications, please consult my Google Scholar page. The list below contains publications I consider the most important or interesting. Journal publications E. Derner, K. Batistič, J. Zahálka, and R. Babuška: A Security Risk Taxonomy for Prompt-Based Interaction with Large Language Models. IEEE Access, vol. 12, pp. 126176–126187, 2024.
- [Teaching](https://zahalka.net/teaching/) - PhD students Graduated dr. Omar Shahbaz Khan (IT University of Copenhagen), graduated September 2022 Information Visualization In 2021, I was the lecturer and course coordinator for the Information Visualization course (5204INVI6Y) of the Master Artificial Intelligence study programme at the University of Amsterdam. In the spirit of open education, I have made the course material
- [Subscribe](https://zahalka.net/subscribe/) - Subscribe to receive blog updates, post notifications, and monthly post summaries by e-mail. * indicates required Email Address *First Name Last Name Marketing PermissionsPlease select all the ways you would like to hear from zahalka.net:EmailYou can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy
- [Subscription successful!](https://zahalka.net/subscription-confirmed/) - Thank you for subscribing to the blog subscriber list. You will receive blog updates, post notifications, and monthly post summaries by email. You can unsubscribe at any time by clicking the unsubscribe button in the welcome email or the unsubscribe link in any email you have received from the subscriber list.
- [Email confirmation](https://zahalka.net/confirm-your-e-mail/) - To complete the subscription, please click the link in the email we just sent you.
- [Privacy policy](https://zahalka.net/privacy-policy/) - Who we are Our website address is: https://zahalka.net. Comments When visitors leave comments on the site we collect the data shown in the comments form, and also the visitor’s IP address and browser user agent string to help spam detection. An anonymised string created from your email address (also called a hash) may be provided
- [Sample Page](https://zahalka.net/sample-page/) - This is an example page. It's different from a blog post because it will stay in one place and will show up in your site navigation (in most themes). Most people start with an About page that introduces them to potential site visitors. It might say something like this: Hi there! I'm a bike messenger

## Categories

- [Science](https://zahalka.net/ai_security_blog/category/science/)
- [CVPR '22](https://zahalka.net/ai_security_blog/category/science/cvpr-22/)
- [CVPR '23](https://zahalka.net/ai_security_blog/category/science/cvpr-23/)
- [AI](https://zahalka.net/ai_security_blog/category/ai/)
- [Security](https://zahalka.net/ai_security_blog/category/security/)