Hi, everyone! If you are getting ready to join me at Unleash, I hope to see you there! I thought it would be useful to share 10 questions with suggested answers and red flags. Feel free to use them in your conversations with vendors. Hope it helps!

See you in Paris, Anita

10 Questions to Ask HR Vendors

I think we can safely say that this year, HR and payroll tech have moved from AI hype to substance. I hardly see vendors make bold, unsubstantiated claims about AI capabilities anymore. But that’s not to say they are always perfectly clear when they talk about AI. Which means, it’s now up to you. As you go to a conference (see you at Unleash!) and evaluate solutions for your organization, you will need to understand what AI can and can’t (yet) do, and how to distinguish between genuine innovation and clever repackaging.

Talking about AI is not always easy, and you might not feel fully confident with all the different topics. As you prepare for Unleash, I want to equip you with three key concepts that will help you navigate vendor conversations, and then I’ll give you 10 questions that can help you quickly understand what’s happening under the hood.

Three key concepts

AI-Native vs. AI-First: An AI-native solution is built from the ground up with AI at its architectural core: the product wouldn't exist without AI. An AI-first solution takes an existing platform and strategically integrates AI capabilities. Both approaches have merit, but vendors must be transparent about which they offer. If a vendor built their platform before 2015, they cannot legitimately claim to be AI-native, regardless of how deeply they've since integrated AI. And even if they built their platform after, it might still be AI-first.

Deterministic vs. Probabilistic AI: This distinction is crucial for understanding where AI is appropriate in HR systems. Deterministic systems produce the same output every time for the same input and follow fixed rules and calculations. Core HR functions like payroll calculations, time tracking, and compliance checks are deterministic because they require absolute accuracy and consistency. Probabilistic AI systems, like large language models, generate outputs based on patterns and probabilities. They produce different answers to the same question. These can be appropriate for talent acquisition, employee engagement analysis, or generating content suggestions, where variation and creativity can be valuable.

EU AI Act Compliance: HR and recruitment solutions are classified as high-risk AI systems under the EU AI Act, which means they must meet strict requirements for transparency, bias prevention, data management, and human oversight when processing the data of European employees. Most obligations take effect in August 2026, though AI literacy requirements are already in force as of February 2025.

How You Use The 10 Questions

Please don't treat this as a checklist to rush through. That’s not how I designed the questions, and it’s also not appropriate for a short conversation on the Expo floor. The checklist is meant as a quick way to determine if a vendor knows their AI stuff and if you should continue to conversation. I have given you 10 questions and I’ve added keywords to describe what you should listen for, and what can point to potential red flags.

Pick the questions most relevant to your evaluation priorities and see what happens when you engage vendors. Don’t just listen to the answers but watch how confidently and specifically the vendor’s staff responds. If they don’t know the answer, will they say so? Will they invite an expert to the conversation? The best vendors will welcome these questions as opportunities to demonstrate their expertise.

Strong vendors will:

• Answer directly and specifically rather than deflecting with buzzwords
• Acknowledge limitations and trade-offs openly
• Provide documentation, case studies, or demos to support claims
• Connect you with existing customers facing similar challenges
• Discuss compliance proactively rather than reactively

You should take notes during conversations and compare answers across vendors. Inconsistencies often reveal where marketing claims diverge from technical reality.

The AI revolution in HR is real, but so is the imperative to implement it thoughtfully, compliantly, and effectively. I hope these questions will help you find partners committed to substance over hype.

10 Critical Questions to Ask HR Tech Vendors

1. Is your solution AI-native or AI-first, and what does that mean for your architecture? Which functions use deterministic logic versus probabilistic AI?

Why this matters: Vendors often use AI-native as a marketing term without understanding its technical meaning. Press them to explain their architecture. An AI-native system has AI embedded in its core data structures, uses model-driven logic rather than rules-based programming, and was designed from inception with continuous learning capabilities. An AI-first system retrofits AI onto existing infrastructure, which is perfectly acceptable, but requires honest disclosure. (Most vendors will be AI-first.) You must also understand which parts of the system are deterministic (following fixed rules for consistent outputs) versus probabilistic (using AI models that may vary). Payroll, time tracking, and compliance calculations must be deterministic. Talent screening or content generation can be probabilistic.

What to listen for: Clear architectural explanation, honest disclosure about when they built the platform, explicit mapping of which functions are deterministic versus probabilistic, and an appropriate separation that ensures compliance-critical calculations never rely on probabilistic AI.

Red flags: Vague answers, claims of being AI-native for products launched before 2022, inability to explain architectural differences, or use of probabilistic AI (like LLMs) for payroll calculations or other deterministic functions.

2. How does your solution comply with the EU AI Act's requirements?

Why this matters: The EU AI Act requires vendors of high-risk AI solutions to ensure transparency, appropriate human oversight, continuous monitoring, and data quality management. Employers must inform candidates and employees about AI use and explain how it functions and how decisions are made. Candidates and employees have the right to request explanations about the role AI played in decision-making. The Act also requires human oversight at key decision points. This means that fully automated HR decisions without human review may violate both the AI Act and anti-discrimination laws.

What to listen for: Specific mentions of risk management systems, documentation procedures, logging capabilities, candidate/employee notification mechanisms, features explaining why and how decisions are made, clear workflows with mandatory human review points, role-based access controls, decision audit trails, and mechanisms that prevent bypassing human oversight.

Red flags: Unfamiliarity with the Act, claims it doesn't apply to them, "black box" AI that can't be explained, claims of "fully automated" HR processes, inability to show where humans review AI recommendations, or suggestions that compliance is solely your responsibility as the customer.

3. How do you prevent, detect, and mitigate bias in your AI models?

Why this matters: Vendors and employers must ensure AI training data is relevant, sufficiently representative, accurate, and without bias to prevent discriminatory outcomes. Bias can creep in through training data, model design, or deployment contexts. Vendors should have systematic approaches to addressing this throughout the AI lifecycle.

What to listen for: Regular bias testing protocols, thoughtful explanation of how the vendor’s diverse training datasets were created, fairness metrics they monitor, third-party audits, and concrete examples of bias they've identified and corrected.

Red flags: Claims that AI is inherently objective, inability to explain their bias testing methodology, or defensive responses to bias questions (“what do you mean by bias?”).

4. What data does your AI use for training, how do you ensure data quality, and what governance and audit capabilities do you provide?

Why this matters: AI systems are only as good as their training data. Outdated, biased, or non-representative data sets produce flawed outputs. You need to understand what data trained the models, how often they're retrained, and whether the training data reflects your workforce and candidate pool. Additionally, high-risk AI systems must maintain technical documentation, log results, and track usage for compliance, accountability, and potential legal challenges.

What to listen for: Specifics about training data sources, data curation processes, how they ensure diversity in training data, data quality checks, retraining schedules, automated logging of AI decisions and confidence scores, immutable audit trails, data retention policies aligned with GDPR, version control for AI models, and easy-to-generate compliance reports.

Red flags: Vague answers about "large datasets," an inability to describe data curation processes, training data that doesn't reflect diverse populations, limited logging capabilities, inability to reconstruct past decisions, or audit trails that can be modified after the fact.

5. How does your AI use our data for training, and how do you ensure data privacy and security?

Why this matters: You want the AI solution to access your company data, like HR policies, procedures, organizational context, to provide relevant, contextualized responses (through techniques like Retrieval-Augmented Generation or RAG). However, your sensitive employee and candidate data must never be used to train the vendor's underlying AI models, which could expose it to other customers or external parties. Understanding this distinction is critical for both functionality and GDPR compliance.

What to listen for: Clear explanation of RAG or similar techniques that allow the AI to reference your data without incorporating it into model training, explicit contractual guarantees that customer data will not train their models, data isolation between customers, encryption standards (in transit and at rest), data residency commitments (especially for EU data), and third-party security certifications (SOC 2, ISO 27001).

Red flags: A confusion vendor explanation about using data for context versus training, vague data usage policies, inability to guarantee your data won't train models used by other customers, resistance to contractual data usage restrictions, or lack of clarity on data storage locations and sub-processors.

6. How does your AI perform across different languages and cultural contexts relevant to our workforce?

Why this matters: AI systems are often trained primarily on English-language data and reflect predominantly Anglo-American cultural norms. In a diverse European workforce, you need AI that understands multiple languages, regional variations (Swiss German vs. German, for example), and cultural nuances in communication styles, workplace norms, and expectations. Poor multilingual performance or cultural bias can lead to discriminatory outcomes and undermine system effectiveness.

What to listen for: Specific languages and dialects supported, performance metrics by language (not just "we support 50+ languages"), examples of cultural adaptation in the system, how they handle code-switching or multilingual documents, training data diversity across languages and regions, and whether human reviewers represent your linguistic and cultural diversity.

Red flags: Claims of universal language support without performance data, significant performance drops in non-English languages, inability to discuss cultural considerations beyond translation, or training data that's primarily English with machine-translated additions.

7. What happens when your AI fails or produces incorrect results? How do you handle edge cases? And how do you ensure deterministic functions remain accurate?

Why this matters: No AI system is perfect. It will make mistakes (just like your current systems). Understanding failure modes, error rates, and recovery mechanisms reveals system maturity and vendor honesty. Edge cases (unusual situations the AI hasn't been trained on) are where most real-world problems emerge. For probabilistic AI (like talent screening), variability is expected and manageable. For deterministic functions (like payroll), any error is unacceptable and potentially illegal. You need to understand how vendors ensure deterministic accuracy while managing probabilistic uncertainty.

What to listen for: Documented error rates for probabilistic functions, confidence thresholds that trigger human review, fallback mechanisms, examples of failures they've addressed, continuous monitoring for anomalies, and absolute guarantees of accuracy for deterministic calculations with validation and testing protocols.

Red flags: Claims of near-perfect accuracy without distinguishing between deterministic and probabilistic functions, lack of error handling processes, inability to discuss system limitations, or any suggestion that payroll or compliance calculations use probabilistic AI.

8. Which underlying AI models does your solution depend on, and what happens if those providers change pricing, availability, or terms?

Why this matters: Many HR tech vendors build their solutions on top of third-party foundation models (OpenAI's GPT, Anthropic's Claude, Google's Gemini, or open-source alternatives). This creates hidden dependencies that can affect your costs, data security, performance, and long-term viability. If the underlying model provider changes terms, raises prices dramatically, or discontinues service, your HR services could be severely impacted.

What to listen for: Transparency about which foundation models they use, whether they have multi-model strategies or can switch providers, how model changes have affected their service, contractual protections they have with model providers, whether they have fallback options, and how quickly they can adapt to model provider changes.

Red flags: Complete dependency on a single model provider without alternatives, inability or unwillingness to disclose which models they use (citing "proprietary" reasons), no contingency plan for model provider changes, or recent disruptions due to provider issues that weren't handled smoothly.

9. Beyond licensing fees, what are the ongoing costs for AI usage, and how do they scale?

Why this matters: Many AI vendors have pricing models that include variable costs: per API call, per user action, per document processed, or tiered usage limits. What seems affordable in a pilot can become expensive at scale. You need complete transparency on total cost of ownership to budget accurately and avoid nasty surprises as adoption grows.

What to listen for: Clear breakdown of fixed vs. variable costs, pricing tiers and what triggers moving between them, examples of typical monthly costs for organizations your size, whether costs increase linearly or exponentially with usage, any included usage limits before overages apply, and price protection or caps for the contract term.

Red flags: Reluctance to discuss pricing beyond base license fees, complex pricing models that are difficult to forecast, significant per-transaction fees that could spiral with adoption, or unwillingness to provide estimates based on your anticipated usage patterns.

10. How long does typical implementation take, and what are the most common reasons deployments fail or get delayed?

Why this matters: The gap between demo and reality can be enormous. A solution that looks brilliant in a controlled vendor demonstration may struggle with your messy real-world data, complex workflows, legacy system integrations, and organizational change resistance. Honest vendors will share realistic timelines and common pitfalls. This transparency reveals maturity and helps you plan appropriately.

What to listen for: Realistic implementation timelines (measured in months, not weeks, for enterprise deployments), specific examples of implementation challenges they've encountered, data quality requirements that often cause delays, change management resources they recommend, typical ramp-up time to full value, and customer references you can contact about their implementation experience.

Red flags: Overly optimistic timelines ("fully deployed in 2 weeks"), inability to discuss implementation failures or challenges, claims that their solution "just works" without significant effort, lack of implementation methodology or best practices, or unwillingness to connect you with customers who've completed implementations.

I give keynotes and workshops on the Future of Work & Pay. At conferences, in-company or virtually. Do you need an interactive session to kick off your event? Please reach out!

This newsletter is free. You can support me by buying my books What's Up With My Pay?, Equal Pay for Equal Work and How to Select Your Next Payroll. Or call me for advice when you are selecting new HR or payroll tech.

Did you enjoy this newsletter? Forward it to someone who might like it!