Skip to main content
PYRAMYD
Blog

Signals · Pipeline

The 2,000-Source Signal Feed: How a Competitor Move Reaches the Graph

The PYRAMYD signal feed draws on 2,000+ registered source feeds across press, changelogs, hiring, pricing pages, and filings, resolves names to nodes, and writes typed events back to the graph; you refresh any competitor you track on demand. This is the engineering that turns raw web noise into auditable graph state.

Jonathan Krasnow · Founder & CEO, PYRAMYDApril 22, 202612 min read
SignalsPipelineProvenanceETL
Editor's noteThis post is part of our research-grounded series. Third-party statistics referenced here are being re-verified against primary sources as part of an ongoing content audit. Where the original source isn't machine-verifiable, we're reframing the claim qualitatively or citing the underlying primary paper directly. Reach out if you spot a citation that should be tightened.
ShareLinkedInXEmail

In Crayon's 2024 State of Competitive Intelligence report, 58% of CI professionals name keeping competitive intel up to date as one of their top two challenges.[1] That is the structural failure of document-driven competitive intelligence. The fix is not better battlecards. It's a signal feed the battlecards are rebuilt from whenever you refresh them.

Battlecards lag reality because they're documents, not queries. The fix is a signal feed that's closer to the source — 2,000+ registered feeds, on-demand refresh, names resolved to nodes, typed event nodes written back to the graph.

PYRAMYD's signal feed draws on 2,000+ registered source feeds[7] across press releases, product changelogs, hiring posts, funding announcements, integration directories, review platforms, regulatory filings, and a long tail of vertical-specific sources. Each source can be refreshed when you ask. The output is typed event nodes in the graph · queryable, citable, dated, and FK-linked to the nodes they describe.

Section 1: What "signal" means in the typed-graph world

In a document-grounded CI system, a "signal" is a unit of text — "Salesforce announced a new pricing tier today." The string is hard to query, hard to deduplicate, and hard to count over time. Hundreds of signals can describe the same underlying event; the analyst has to do the de-duplication mentally.

In a typed-graph system, a signal is a structured event with referential integrity:

  • Event type · drawn from a closed vocabulary: product, funding, security, legal, hiring, partnership, pricing, acquisition, and more (57 types in use on 8 Oct 2026).
  • Subject node · the Vendor, Product, Category, or Industry the event is about · resolved to the canonical node ID, not a string.
  • Source citation · URL, retrieval timestamp, source confidence, source authority score.
  • Structured payload · the fields specific to the event type (a Release has version + release notes URL; a FundingRound has round size + valuation + lead investor; etc.).
  • Discovery timestamp · when the signal pipeline first saw it (which is what powers the "new this week" feeds and the diff-since-last-load comparisons).

Once signals are typed events with referential integrity, you can count them ("Salesforce signals in Q3 2026"), aggregate them ("pricing signals per vendor in CRM Software, by quarter"), filter them ("funding rounds led by Sequoia in the Cloud Infrastructure category, last 90 days"), and chart them. None of which the document layer supports.

Section 2: The pipeline in five stages

Stage 1: Source discovery

Each source is registered with a manifest — base URL, fetch method (RSS / API / scrape), cadence, expected payload shape, source authority score, and a per-source retry policy. Requests identify the PYRAMYD crawler in their headers.

Stage 2: Fetch + content-hash dedup

The scoring step is content-hashed, an idea adapted from Stripe's idempotency keys, which let a repeated request be recognised and not processed twice.[3] When the same text comes in again, the cached score is reused instead of calling the model again. That is part of what lets us register thousands of sources without proportional cost.

Stage 3: LLM extraction

New payloads (cache miss) get extracted by a frontier-tier LLM into the typed-event schema. Extraction prompts are per-event-type — a release-extraction prompt is different from a funding-extraction prompt. The model returns a structured JSON object that gets validated against the schema; failures get logged and routed to a human queue for schema-extension review.

Stage 4: Resolving names to nodes

A hard stage, and one the Microsoft, Google, Facebook, eBay, and IBM retrospective on industry-scale knowledge graphs names among its shared challenges.[2] A press release that says "Salesforce announced..." has to resolve to the canonical Salesforce vendor node, not a duplicate "Salesforce, Inc." node. The resolver works in tiers:

  • Exact match. A name that matches a known node or one of its aliases resolves straight to it.
  • Fuzzy text match. If there is no exact match, a trigram similarity match catches near misses such as spelling variants, with a minimum score before it counts.
  • No guessing. If no tier finds a confident match, the name stays unlinked rather than being attached to the wrong node.

Stage 5: Typed write into the graph

Resolved events are written as signals with links (foreign keys) to the company, product, category, industry, country, or role they concern and to the source they came from, plus two dates: when it happened and when we detected it.

2,000+

registered source feeds

403

sources with a refresh schedule set up

~180

signals a day, 7 days to 4 Oct 2026

Section 3: The source manifest — what's in the catalog

The 2,000+ sources are not uniform. They're weighted by signal-to-noise ratio and by authority, and each one is classified by the actor that generates its signals. The catalog by category, counted on 8 October 2026 (2,173 active sources):[7]

  • Governance · 553 sources · regulators, standards bodies, courts, and public registers.
  • Observers · 411 sources · press, analysts, and commentators.
  • Commercials · 322 sources · pricing, packaging, and commercial terms.
  • Ecosystem · 276 sources · partners, marketplaces, and integrations.
  • Technology · 251 sources · releases, changelogs, and product documentation.
  • Talent · 149 sources · job boards and careers pages.
  • Customers · 140 sources · reviews, communities, and customer stories.
  • Capital · 57 sources · funding, filings, and M&A.
  • Marketing · 14 sources · campaigns and announcements.

Section 4: How operators actually consume the feed

In PYRAMYD, signals are first-class: battlecards are generated from the graph and its signals, then your team reviews and edits them. Rebuild one whenever you need it.

Section 5: The audit log — every signal traceable back to source

Every signal in the graph carries its source URL, its link to the source it came from, and its dates (when it happened and when we detected it); most also record the model that scored them. Click a signal and you can open the source it came from. This is what lets the feed answer an auditor — and what makes it actually trustworthy when a CFO asks "where does this number come from?"

It also matters for the model layer. Every signal an APEX answer uses is listed in its Sources panel, with a link back to the original where one exists. If an APEX answer says "Salesforce shipped a new pricing tier this week," the user can open the source and land on the press release the extractor read. The audit chain is what makes the LLM's claim checkable rather than "trust me."

Where this lands for PYRAMYD customers

The signal feed is the data layer PYRAMYD ships underneath the typed graph — 2,000+ registered source feeds, every item linked to the companies it touches, with a full audit chain back to source. Each refresh you run is what brings the 360 pages, the battlecards, and the APEX answers up to date, instead of leaving them on last quarter's document.

The deeper claim: signals were always the right primitive. Document-centric CI made them second-class because the only available substrate was text. With a typed graph in the middle, signals can be first-class — counted, aggregated, filtered, charted, audited. The battlecards in the workspace are generated from the graph and its signals, and rebuilt whenever you need them. That narrows the refresh gap by construction.

ShareLinkedInXEmail

References

  1. [1]Crayon, 2024 State of Competitive Intelligence (opens in a new tab) · Crayon's annual survey of CI professionals: 58% name keeping competitive intel up to date as one of their top two challenges.
  2. [2]Noy, N. et al., Industry-scale Knowledge Graphs: Lessons and Challenges, Communications of the ACM, 62(8), 36-43 (Aug 2019) (opens in a new tab) · Microsoft, Google, Facebook, eBay, and IBM compare how they built their knowledge graphs; resolving the same real-world thing across sources is among the shared challenges.
  3. [3]Leach, B., Designing robust and predictable APIs with idempotency (Stripe blog, 22 Feb 2017) (opens in a new tab) · Idempotency keys let a repeated request be recognised and not processed twice; the signal feed applies the same idea to fetched content.
  4. [4]Google SRE Book, Chapter 22: Addressing Cascading Failures (O'Reilly, 2016) (opens in a new tab) · Reference for rate limiting, load shedding, and retry budgets: how a fetch pipeline avoids hammering sources when one cohort spikes.
  5. [5]Lewis, P. et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020 (opens in a new tab) · The pattern analysts use after extraction: retrieval over the structured signal events.
  6. [6]Edge, D. et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization, arXiv:2404.16130 (Apr 2024) (opens in a new tab) · Graph-grounded retrieval: the architectural pattern signals are written into, typed event nodes with edges back to the nodes they describe.
  7. [7]PYRAMYD signal source catalog, production database (read 8 Oct 2026) · 2,720 registered sources, 2,173 of them active; 403 have a refresh schedule set up.

See it with your own product.

Book a demo. We'll connect your product to its market, competitors, customers, and deals, and show APEX agents answering from that evidence with every source listed.