Taking off the blinders: LLMEx for machine learning explainability. Joshua Cason Staff Software Engineer, Demandbase In the modern enterprise, stakeholders acknowledge the value of predictive models, but they struggle to trust outputs that they cannot interpret. Blindly trusting opaque numbers is error fraught, especially in high-stakes sales pipelines where a misunderstood signal can lead to a missed opportunity or wasted effort. Data always has "blind spots," and without visibility, its stakeholders are operating in the dark. The solution: LLMEx. To bridge this gap, Demandbase, Inc. developed LLMEx, a framework designed to make complex model outputs human-readable without sacrificing mathematical integrity. Its core philosophy is simple: by articulating the rationale behind a predictive score in plain English, Demandbase, Inc. provide stakeholders with the context necessary to fill in the blind spots. When users understand what tipped the model in a specific direction, they are no longer just consumers of a prediction; they become active partners in the process. This transparency empowers them to apply human intuition to adjust, refine, or even reject the model's advice on a case-by-case basis. Under the hood. At its heart, LLMEX is about extracting meaningful signals from model output and converting them into actionable narratives. Building on feature importance techniques like SHAP which provide scores on its decision factors, LLMEX translates those scores into intuitive, plain-English rationales. However, generating natural language is not enough - it must be accurate. To ensure integrity, LLMEX employs a dual-model training architecture: * Rationale Generation: The first model call generates a human-readable explanation based on the underlying feature ranking. * Reconstruction Verification: The second model call attempts to reconstruct the feature rank ordering solely from that rationale. Crucially, this second model cannot see the raw feature rank inputs. This "double-check" process acts like an autoencoder pattern: it encourages the rationale generator to encode the ranking faithfully. If the explanation is hallucinated or lacks depth, the second model (call it the Judge) will fail to reconstruct the feature rankings. This ensures the output is both trustworthy and grounded in the actual feature importance scores. Engineering for confidence: DSPy. Building complex LLM systems can be brittle and time consuming, especially when relying on traditional "prompt engineering." To iterate more quickly and ensure its explanations are reliable, Demandbase, Inc. optimize its LLMEx system with DSPy. DSPy provides the software abstractions Demandbase, Inc. need to organize multi-module LLM systems and jointly optimize their prompts. This approach is often as powerful as fine-tuning but offers more flexibility, for example, by removing the need for complex GPU infrastructure. Moreover, it is bitter-lesson pilled: To benefit from the latest frontier intelligence, Demandbase, Inc. just "re-compile" LLMEx using the newer LLM model. The bottom line, however, is that by testing its explanations over a large, representative test set using DSPy, Demandbase, Inc. can deploy LLMEx with statistical confidence. Demandbase, Inc. is no longer guessing whether its model is hallucinating; Demandbase, Inc. is measuring its performance against ground truth. Experiment. Below I describe experiments I did to illustrate the power of this method to increase trust in the English outputs. This is a simplified setup compared with its production system with many variables removed. A discussion follows below about those differences. What I show is that prompt optimization can drastically improve over the use of a naive prompt. Setup. I began with a sample of around 1,900 accounts from Demandbase's own data. Demandbase, Inc. run its proprietary Pipeline Predict model across all accounts on a daily basis. The results of that prediction include feature importance structures stored in its database systems. Demandbase, Inc. only look at the Likely and Highly Likely accounts for LLMEx. I first selected a holdout of 50 accounts at random for the test set. As a baseline, I ran the LLMEx simulation program without DSPy optimization. Then, starting at 200 training samples, I trained the model (i.e., optimized the prompt and in-context examples). After each training session, I saved that model and then doubled the amount of training examples until I reached the full training pool at around 1,600 samples. I also provide a fixed validation set to the optimizer of 180 samples. The metric for grading the explanations (Figure 1) is a simple pairwise accuracy augmented by a coverage and purity score. These are analogous to recall and precision and check that the Judge found all and only the feature names provided. These are combined into an F1 score and multiplied times the pairwise accuracy for a final metric. I use this metric in DSPy's MIPROv2 optimization (a.k.a., compilation) and use it to compute test set accuracy after training (or before in the baseline case). Figure 1. Breakdown of the evaluation metric. Results. As you can see below, with each training round the ability of the Judge to determine the feature ranking from the text increases for a total increase of +14 from 73% to 87%. This confirms the hypotheses that (1) optimized prompts with few-shot examples indeed improve the power of the model to communicate the required information, assuming the LLM is a suitable proxy for human readers, and that (2) increased training increases that accuracy similar to traditional machine learning. Figure 2. Progressive accuracy as training sample size is increased. Cheating mitigation. One reality that has set in during the new era of LLM evals is the specter of cheating. Simpler models like SVM or Random Forest don't have a motivation to cheat or any motivations at all. But LLMs, whether Demandbase, Inc. call it a motivation or not, have a documented history of cheating with the clear intention of boosting their apparent accuracy. While this phenomenon typically crops up during agentic evaluations where LLMs have the ability to observe and change their environment, its simpler task can still fall victim. During the product development cycle, I noticed blurbs where the encoder model would insert the feature importance numbers directly next to the name of the feature, giving the Judge what it needed to perform a boosted evaluation. While I can imagine other ways a model could cheat, I took the most straightforward approach to mitigating this behavior. Whenever a feature importance value is detected in the text, that sample is immediately given a score of zero. The quality of these texts is highly important. I want its customers to be sharpened over time by providing accurate and insightful glimpses of the impact of their go-to-market motions. Cheating mitigation is an important tool in that endeavor. Cheating mitigation is effectively hallucination mitigation, because it ensures its statistical rigor isn't hindered by cover-ups. Simulation vs production. The simulation above is meant to demonstrate the power of prompt optimization to increase the faithfulness of its predictive explainability. Much more goes into its process on the platform. Additional measures Demandbase, Inc. take during the training procedure, for example, include penalties for extra long blurbs and markdown quality misses. Moreover, since Pipeline Predict is focused on timely market activity, firmographic information about the target account, despite often being important, is included towards the bottom of the blurbs. Conclusion. Ultimately, transparency isn't just a byproduct of good design; it is the catalyst for AI adoption. By using frameworks like DSPy to rigorously optimize its rationale generation, Demandbase, Inc. has moved beyond simple interpretability to verifiable trust - demonstrating that even in complex systems, Demandbase, Inc. can quantify and improve human-readable outputs by a significant margin. This doesn't just demystify the 'black box'; it transforms AI from an inscrutable oracle into an active partner. By merging the statistical rigor of traditional machine learning with the generative fluency of LLMs, Demandbase, Inc. is not just improving a tool - Demandbase, Inc. is redefining the partnership between data and human intuition, ensuring that its decisions are grounded in evidence rather than blind faith.

PathFactory review: features, pricing, Content Intelligence, and competitors. PathFactory is a B2B content intelligence and engagement platform designed to help marketing teams understand how prospects consume content, then use that behavior to accelerate pipeline. Instead of treating content as a static library of PDFs, webinars, videos, and case studies, PathFactory turns those assets into trackable, personalized buying journeys. For revenue teams struggling to connect content performance with sales readiness, it can be a powerful layer between marketing automation, CRM, and website engagement. TLDR: PathFactory is best for mid-market and enterprise B2B companies that rely heavily on content marketing, demand generation, and account-based marketing. Its strongest value is showing who consumed what, for how long, and what they should see next. For example, a software company might discover that prospects who spend more than 15 minutes in a product comparison track convert to demo requests at a 28% higher rate than casual visitors. Pricing is quote-based, so it is usually a better fit for teams with mature marketing operations and a clear content strategy. What is PathFactory? All Heading [show] PathFactory helps marketers create curated content experiences and measure real engagement beyond simple clicks or form fills. A person clicking a whitepaper download link tells you very little. Did they read it? Did they watch the related webinar? Did they spend 30 seconds or 30 minutes learning about your product category? PathFactory answers those questions by tracking content consumption behavior. It can show which assets are attracting attention, which topics are moving buyers forward, and which accounts are displaying high-intent activity. This makes it especially useful for B2B organizations with long sales cycles, multiple stakeholders, and large content libraries. Key features of PathFactory. PathFactory combines content delivery, personalization, analytics, and sales enablement into one platform. Its core features are built around making content easier to consume and easier to measure. * Content Tracks: Marketers can group related assets into guided journeys, such as "Beginner's Guide," "Product Evaluation," or "ROI Justification." These tracks encourage binge-style consumption rather than one-off downloads. * Content Intelligence: The platform tracks engagement signals such as time spent, assets viewed, sequence of consumption, topic interest, and repeat visits. * Personalized Experiences: Visitors can be directed to relevant content based on campaign, industry, persona, account, funnel stage, or behavior. * Account-Based Marketing Support: Teams can create account-specific content hubs and monitor engagement across buying committees. * Sales Insights: Sales teams can see which prospects are actively consuming content and use that context for follow-up conversations. * Integrations: PathFactory commonly integrates with platforms such as Salesforce, Marketo, Eloqua, HubSpot, Demandbase, 6sense, and other marketing or revenue tools. * Reporting and Attribution Support: Marketing teams can evaluate which content assets influence pipeline, opportunities, and revenue. Content Intelligence: PathFactory's biggest strength. The phrase content intelligence can sound vague, but in PathFactory's case it refers to collecting and analyzing behavioral data from content experiences. Instead of judging content by downloads alone, marketers can look at deeper indicators such as engagement time, topic clusters, asset sequencing, and buying-stage patterns. For example, if prospects from healthcare accounts repeatedly consume content about compliance, data security, and implementation timelines, the marketing team can create a dedicated healthcare track. Sales can then prioritize accounts where multiple stakeholders have spent meaningful time with related assets. This is much more useful than simply knowing that someone opened an email. PathFactory's content intelligence can also expose gaps in a content strategy. If top-of-funnel guides receive plenty of views but product comparison assets receive little engagement, the issue may be poor promotion, weak content quality, or a mismatch between buyer expectations and available resources. In that sense, the platform is not just a reporting tool; it can guide future content planning. User experience and ease of use. PathFactory is built for marketing teams, but it is not a lightweight plug-and-play tool. Users typically need a structured content library, clear campaign goals, and some marketing operations support to get the full benefit. The interface is generally focused on building experiences, managing assets, segmenting audiences, and analyzing behavior. Teams that already use marketing automation and CRM systems will likely find PathFactory easier to adopt because the platform's value increases when connected to lead scoring, campaign reporting, and sales workflows. However, smaller teams with limited content or minimal operations support may find the platform more sophisticated than necessary. PathFactory pricing. PathFactory does not usually publish standard pricing on its website. Like many enterprise B2B marketing platforms, it uses a custom quote-based pricing model. The final cost typically depends on factors such as company size, number of users, traffic volume, required modules, integrations, implementation needs, and contract terms. Because pricing is not transparent, buyers should prepare for a sales-led evaluation process. It is wise to ask about implementation fees, support levels, data limits, onboarding timelines, integration costs, and renewal terms. Teams should also clarify whether features such as advanced analytics, ABM capabilities, or sales enablement tools are included in the quoted package or priced separately. In general, PathFactory is most likely to suit organizations that can justify the investment through improved conversion rates, better sales prioritization, stronger content ROI, or more efficient campaign performance. If your company has hundreds of content assets and a long buying cycle, the economics may make sense. If you only publish a few blog posts and occasional ebooks, it may be more platform than you need. Pros and Cons. | Pros | Cons | | Strong content engagement analytics | Pricing is not publicly available | | Useful for ABM and demand generation | Best suited for teams with mature content operations | | Helps sales teams identify engaged prospects | May require setup and integration work | | Encourages deeper content consumption | Value depends on having enough quality content | PathFactory competitors and alternatives. PathFactory competes with several platforms in the content experience, personalization, ABM, and buyer engagement categories. The best alternative depends on whether your priority is content hubs, website personalization, interactive content, intent data, or sales enablement. * Uberflip: A well-known content experience platform focused on creating personalized content hubs and destination pages. It is often compared directly with PathFactory, especially for demand generation teams. * Folloze: Strong in personalized buyer journeys and ABM content experiences. Folloze is often used by enterprise sales and marketing teams targeting specific accounts. * Hushly: Offers content engagement, lead conversion, and personalization tools with an emphasis on reducing friction in content access and improving conversions. * ON24: Best known for webinars and digital events, but also offers engagement data and content experiences around event-driven marketing. * Seismic: More sales enablement-oriented than PathFactory, but relevant for teams focused on giving sales reps the right content and measuring usage. * 6sense and Demandbase: These are broader ABM and intent data platforms. They may overlap with PathFactory in account engagement insights, though their core focus is different. Who should use PathFactory? PathFactory is a strong choice for B2B companies that depend on educational content to move buyers through complex decision cycles. It is particularly useful for SaaS companies, technology vendors, professional services firms, cybersecurity providers, healthcare technology companies, and financial technology brands. The ideal customer has a sizable content library, multiple audience segments, and a need to prove how content influences revenue. If your marketing team frequently asks, "Which assets are actually helping sales?" or "Which accounts are truly engaged?" PathFactory can provide more meaningful answers than basic web analytics. However, companies should not expect software alone to fix weak content. PathFactory can reveal behavior, improve presentation, and connect content to pipeline signals, but it still depends on relevant messaging, strong assets, and alignment between marketing and sales. Final verdict. PathFactory is a sophisticated platform for turning content into measurable buyer experiences. Its biggest advantage is its ability to go beyond clicks and downloads by showing real consumption patterns. For enterprise and growth-stage B2B teams, this can improve lead scoring, accelerate sales conversations, and expose which topics truly influence buying decisions. The main drawback is accessibility: pricing is quote-based, and the platform is best suited for organizations with enough content, traffic, and operational maturity to benefit from advanced analytics. Still, for teams serious about content-driven revenue, PathFactory is one of the more compelling options in the content intelligence market. I'm Ava Taylor, a freelance web designer and blogger. Discussing web design trends, CSS tricks, and front-end development is my passion.