AI systems are more and more expected to take autonomous decisions, often well hidden from plain view. This is not necessarily without danger. In this talk I'll give a high-level overview of some common pitfalls in AI-powered applications, that require more thought than many AI gurus would admit. On the menu: bias and fairness, confounding variables, adversarial attacks, ethics, explainability, ... As this presentation was given at a security-focused event, special attention is given to security concerns for individuals and society, such as AI-augmented phishing and disinformation campaigns.