AI Search and Knowledge Intelligence

Legacy search largely optimised for keyword matching. AI Search adds semantic signals and richer ranking controls so “VPN drops” can still find “Remote access troubleshooting” when the vocabulary differs.

Indexes are source-scoped. The most common failure is indexing too much (noise + leakage risk) or too little (empty results).

Architecture rule: treat AI Search like a production service. Define sources, ranking, analytics, and a lifecycle for improvements — not a one-time configuration.

NLU is useful when you have synonyms and ambiguity. It complements embeddings by normalising common patterns and extracting entities (app, device type, location).

Avoid overfitting: NLU intent models require maintenance. If you don’t maintain it, users will drift and recall will collapse.

Design choice: use NLU for high-volume intents (password, VPN, email, access). Do not attempt to model every long-tail question — let semantic retrieval handle the tail.

Embeddings map text into a vector space. When a user asks a question, AI Search retrieves content whose embeddings are closest — not just content sharing the same words.

Semantic similarity fails when content is too generic or when your KB is inconsistent. Fix with better article structure and boosting, not by disabling semantics.

Architect advice: keep a “golden set” of 50–100 queries with expected results. Use it as regression tests when you change sources or ranking.

Think in layers: retrieve candidates (semantic + lexical), then re-rank. Tuning happens mostly in re-ranking.

Three levers you control: boosting, recency weighting, and profile scoping.

Do not use recency to hide low-quality docs. Fix quality. Recency is a tie-breaker, not a cleanup strategy.

Federation requires two decisions: what to connect and who is allowed.

Priority matters: when internal KB and external wiki conflict, decide which wins by portal/persona — otherwise you train users to distrust search.

Architecture rule: federation is not a replacement for knowledge governance. It is an amplifier — it amplifies both good and bad content.

High-yield metrics: zero-result rate, click-through on top result, and abandonment (search then ticket create).

Use analytics to drive knowledge backlog. Assign owners and SLAs like any product backlog.

Measure improvements against a stable baseline — avoid changing ranking and KB simultaneously without tracking which change helped.

Step 1: Enable AI Search features on your PDI (availability depends on instance/build).

Step 2: Configure search sources (start with one KB + catalog).

Step 3: Create a portal profile and test queries with different roles.

Step 4: Tune boosting and recency; rerun golden query set.

Step 5: Capture analytics — zero results, clicks — and iterate.

Migration risks: different ranking, different source scope, and user expectation shifts. Manage by A/B testing before full cutover.

Define success criteria: improved top-1 click-through, reduced zero results, and stable or improved deflection. If deflection rises but wrong-result complaints spike, stop.

Operationally: keep legacy available as fallback during pilot. Make it easy for users to report bad results — that feedback is gold.

Source material matters: the best drafts come from clean resolution notes and well-structured work notes. Garbage tickets produce garbage knowledge.

Draft pipeline: resolved ticket → draft article → SME review → publish → monitor analytics. Do not publish auto-drafts straight to production portals.

Optimise for reusability: a knowledge article should answer a repeatable problem, not describe a one-off incident timeline.

Gap signals: zero-result queries, high abandonment, and repeated escalations after search.

GenAI helps by proposing article outlines and titles for the top gaps — but the important part is the backlog and ownership model.

Treat knowledge gaps like product bugs with SLAs. If 'MFA reset' has zero results, that's an outage of self-service.

Structural signals: clear title, prerequisites, ordered steps, and expected outcomes. Usage signals: helpfulness, time on page, and repeat search after view.

Accuracy is hardest: the platform can flag likely issues (too old, contradictory, low success), but human validation is still required for high-risk procedures.

Use score thresholds: below X → requires review; above Y → eligible for automation and self-service flows.

High-risk domains (security, HR policy, finance) need shorter review cycles. Low-risk how-to content can be reviewed less often.

AI can recommend retire/refresh based on age + low success + contradiction signals. Humans approve retirements for auditability.

Link lifecycle to product releases and tool changes — knowledge should update when the workflow changes.

A structured template prevents the common failure: articles that say “Reboot fixed it” without stating what rebooted, why, and what to do if it fails.

Use extraction for repetitive categories only — where the same fix applies. For one-off incidents, extraction creates noise.

Pipeline discipline: only extract from tickets with high-quality resolution notes and successful closure (low reopen rate).

Tagging improves both lexical and semantic retrieval by adding clean signals that embeddings alone can’t infer reliably (audience, risk, region).

AI can suggest tags and categories from content. Human reviewers approve for high-risk domains to prevent misclassification.

Define a controlled vocabulary for abbreviations and product names — don’t let tags proliferate into chaos.

Machine translation works best for procedural IT steps; it is risky for HR and legal policy where wording matters.

Maintain locale-specific variations: the same “leave policy” differs by country. Translation is not localisation.

Use analytics per locale: zero-result and abandonment patterns differ by language. Treat each locale as a product.

The flywheel converts operational exhaust (queries, tickets, feedback) into better content and better retrieval. Without it, search quality decays and GenAI becomes untrustworthy.

The key is ownership: knowledge manager + portal owner + AI Search admin + SME guild. No owner means no flywheel.

Instrument everything: when a user searches, what did they click, did they create a ticket, did they succeed? Those signals drive iteration.

Profiles segment by surface and persona. Each profile can point at different sources and ranking preferences.

Security implication: HR profile can include HR KB; employee IT profile should not. Profiles plus ACLs form the retrieval boundary.

Practical rule: create the minimum set of profiles that reflect real differences. Too many profiles become unmaintainable.

Boost dimensions include source type (KB vs record), KB base, article category, recency, and popularity signals.

Use boosting to enforce governance: official policy docs should outrank personal wiki pages on compliance topics.

Anti-pattern: boosting everything. If all sources are boosted, nothing is. Keep boosts scarce and justified.

Portal blocks should prioritise “do the thing”: reset password, request access, start chat — not just list documents.

Agent workspace blocks should prioritise speed: similar incidents, KB snippets, and recommended fix steps.

Design principle: show fewer, better blocks. Too many blocks create choice paralysis and reduce deflection.

Maintain a synonym dictionary per domain and locale.

Expansion should be governed: careless synonyms can increase false positives and surface wrong policies.

Use analytics to propose new synonyms: repeated “no results” terms become expansion candidates.

Signals: role (employee vs agent), device fleet (Windows/Mac), region, assigned apps, and history. Use the minimum signals needed.

Transparency matters: users should see why a result was recommended (e.g., “Windows instructions”). Hidden personalisation can look like inconsistency.

Governance: never personalise into restricted knowledge. ACL rules still dominate.

Offline eval: golden query set and expected results. Online eval: A/B testing on traffic and funnel metrics.

Use top-k metrics: top-1 accuracy is ideal; top-3 recall is acceptable when UI guides users. Zero-results must trend down.

Add qualitative audits: sample 20 queries weekly and judge result usefulness — analytics can miss “technically clicked but wrong.”

Use platform search APIs where available instead of replicating ranking in custom code. Keep profiles and sources centralised.

Security rule: never bypass ACLs. Custom experiences must run as the user, not as admin.

UX rule: keep result types consistent with standard search so users recognise what they’re seeing.

Start by reducing scope: fewer sources, smaller indexes, and tighter profiles. Avoid federating to slow external systems for high-volume portals.

Tune retrieval: limit top-k and chunk sizes; demote generic content that explodes candidate sets.

Treat performance as an SLO: define p95 latency targets per portal, and monitor continuously.

Start with category segmentation: password resets and access requests behave differently than outage incidents. Each category has different deflection ceiling.

Include hidden costs: knowledge maintenance, search tuning, and governance. AI doesn’t remove ops; it changes ops.

Use conservative ranges and publish assumptions. Overpromised deflection destroys credibility and kills future funding.

Pattern: user question → AI Search top candidates → GenAI answer grounded in sources → action cards (catalog, reset flow, contact).

Grounding policy: answers must link to KB or catalog items. If no source exists, the assistant should escalate instead of improvising.

Portal design matters: show the answer and the next action on one screen. Minimise clicks between search and resolution.

Before create: portal answers and action flows. During create: suggest KB and detect duplicates. After create: PI routing and Now Assist summaries improve agent productivity, not deflection.

Success rates differ: “password reset” has high deflection ceiling; “network outage” does not. Tie funnel interventions to category.

Design for honesty: do not block ticket creation behind endless chat. Make escalation easy when needed.

Proactive works when you have strong patterns and clean content: top intents, stable procedures, and clear action paths.

Noise kills it: irrelevant popups train users to ignore the portal. Use strict thresholds and limit to top intents.

Pair with accessibility: proactive suggestions must be keyboard and screen-reader friendly.

Core metrics: search sessions, zero results, click-through, ticket create rate after search, and deflection by category.

Attribution pitfalls: users may search then call; one user may create ticket later; a successful session may still create ticket for compliance reasons.

Reporting pattern: show ranges, confidence, and negative signals (false deflection, wrong answers). Mature programs report risks alongside wins.

Fallback triggers: no sources, low confidence, policy-sensitive topic, or explicit user frustration.

Context handoff: include the user’s query, what they clicked, and the top retrieved articles — so agents don’t restart from zero.

Design rule: never trap users in a loop. Provide “create ticket now” option early.

Run the loop: analytics → identify top failures → ship 5 fixes → re-measure. Consistency beats occasional big rewrites.

Separate content fixes from ranking fixes. Content fixes improve truth; ranking fixes improve findability. You need both.

Build champions: portal owners, knowledge managers, and service desk leads. If operations doesn’t own it, it will die after launch.

Architecture: profile-scoped AI Search + KB flywheel + portal action blocks + fallback handoff summary + analytics dashboards.

Before metrics: high zero results, low click-through, high ticket creation after search. After metrics: improved top-1 click-through, reduced zero results, higher deflection in target categories.

Lessons: don’t skip knowledge hygiene; don’t launch without analytics; don’t claim universal deflection; invest in fallback UX.