Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 83 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ export default function App() {
| `useNewsCount(params)` | `/1/count` | Aggregate counts (requires `from_date`, `to_date`) |
| `useCryptoCount(params)` | `/1/crypto/count` | Aggregate crypto counts |
| `useMarketCount(params)` | `/1/market/count` | Aggregate market counts |
| `useNewsStream(registrationId)` | `wss://ws.newsdata.io/ws/event` | Real-time stream ([below](#real-time-news-websocket)) |

Every hook has the same shape:

Expand Down Expand Up @@ -111,6 +112,85 @@ useLatestNews({ country: ['us', 'gb'], language: 'en', size: 20 });
Inline param objects are safe — the hook compares by **value**, not reference,
so re-renders only re-fetch when the values change.

## Real-time news (WebSocket)

`useNewsStream` opens a real-time connection for a **registered query** and
accumulates the matching articles. Register the query first — the returned
`registration_id` identifies it from then on:

```jsx
import { useEffect, useState } from 'react';
import { useNewsDataClient, useNewsStream } from 'newsdataapi';

function BitcoinTicker() {
const client = useNewsDataClient();
const [registrationId, setRegistrationId] = useState(null);

useEffect(() => {
client.websocketRegister({ q: 'bitcoin', language: 'en' })
.then(({ results }) => setRegistrationId(results.registration_id));
}, [client]);

// A falsy id defers connecting until registration resolves.
const { articles, error, isConnected } = useNewsStream(registrationId);

if (error) return <p>Stream error: {error.message}</p>;

return (
<>
<p>{isConnected ? 'live' : 'connecting…'}</p>
<ul>
{articles.map((a) => <li key={a.article_id}>{a.title}</li>)}
</ul>
</>
);
}
```

The hook returns `{ articles, latest, error, isConnected }`. Articles
accumulate **newest first** and are capped at `maxArticles` (default 100) so a
long-lived stream can't grow without bound. The connection opens on mount,
closes on unmount, and reconnects through a capped exponential backoff.

```jsx
const { articles, latest } = useNewsStream(registrationId, {
enabled: true, // defer connecting when false
maxArticles: 100, // cap on retained articles
reconnect: true, // auto-reconnect on transient drops
});
```

`error` is set for a permanent rejection — bad API key or unknown `registration_id`,
exhausted API credits, or too many simultaneous devices — which surfaces as `NewsdataWebSocketAuthError` and is **not** retried.

The server always accepts the handshake and then closes with code **1008** when
the connection is refused, carrying one of three reasons: `invalid credentials
or registration not found`, `api limit reached`, or `device limit reached` (more
than 5 devices on one `registration_id`). Every other close code — including
`1013` (`send timeout`, meaning the client read too slowly) — is transient and
reconnects.

**Each delivered article consumes 1 API credit per connected device.**

Transient drops reconnect silently and leave `error` null.

Managing registered queries goes through the client:
`websocketRegister(params)`, `websocketFetch()`, and
`websocketDelete(registrationId)`. Registering an identical query twice
rejects with a `NewsdataApiError` whose `statusCode` is 409; the existing id is
at `err.responseBody.results.registration_id`.

Outside React, use the same `NewsDataApiWebSocket` class the hook wraps:

```js
import { NewsDataApiWebSocket } from 'newsdataapi';

const ws = new NewsDataApiWebSocket(client);
for await (const response of ws.stream(registrationId)) {
console.log(response.results);
}
```

## Provider

Two ways to wire it up:
Expand Down Expand Up @@ -163,7 +243,9 @@ NewsdataError (catch-all base)
│ ├── NewsdataAuthError (401 / 403)
│ ├── NewsdataRateLimitError (429; .retryAfter)
│ └── NewsdataServerError (5xx)
└── NewsdataNetworkError (.cause)
├── NewsdataNetworkError (.cause)
└── NewsdataWebSocketError (real-time stream)
└── NewsdataWebSocketAuthError (policy-violation close 1008)
```

Validation errors are thrown **before** the request is sent (no API quota
Expand Down
84 changes: 73 additions & 11 deletions src/core/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
import {
BASE_URL,
ENDPOINTS,
ENDPOINT_METHODS,
RESULTS_OPTIONAL,
WS_NEWS_TYPE,
DEFAULT_REQUEST_TIMEOUT,
DEFAULT_MAX_RETRIES,
DEFAULT_RETRY_BACKOFF,
Expand Down Expand Up @@ -124,6 +127,67 @@ export class NewsDataApiClient {
return this.#request('sources', validated);
}

// ---- real-time query management ---------------------------------------

/**
* Register a real-time WebSocket query. POST /1/websocket/register
*
* Takes the familiar filter names (`q`, `country`, `language`, `domain`, …);
* no date or paging filters apply, since a registered query matches news as
* it is published. The new query's id is at `results.registration_id` — pass
* it to `NewsDataApiWebSocket#stream`.
*
* Registering an identical query twice rejects with a `NewsdataApiError`
* whose `statusCode` is 409; the existing id is at
* `err.responseBody.results.registration_id`.
* @returns {Promise<object>}
*/
websocketRegister(params = {}) {
const { rawQuery = null, ...rest } = params;
const validated = validateParams('websocket_register', rest, rawQuery);
validated.news_type = WS_NEWS_TYPE;
return this.#request('websocket_register', validated);
}

/**
* List the account's registered real-time queries. GET /1/websocket/fetch
* One entry per query at `results.queries`.
* @returns {Promise<object>}
*/
websocketFetch() {
return this.#request('websocket_fetch', {});
}

/**
* Delete a registered real-time query. DELETE /1/websocket/delete
* @param {string} registrationId
* @returns {Promise<object>}
*/
websocketDelete(registrationId) {
if (typeof registrationId !== 'string' || registrationId === '') {
throw new NewsdataValidationError(
'registrationId must be a non-empty string',
'registration_id',
);
}
return this.#request('websocket_delete', { registration_id: registrationId });
}

/** The API key, for the WebSocket handshake URL. @internal */
get apiKeyForWebSocket() {
return this.#apiKey;
}

/** The configured fetch, reused by the WebSocket handshake probe. @internal */
get fetchForWebSocket() {
return this.#fetch;
}

/** Forward a log line from the WebSocket layer. @internal */
logForWebSocket(level, message) {
this.#log(level, message);
}

// ---- dispatch ---------------------------------------------------------

/**
Expand Down Expand Up @@ -179,16 +243,17 @@ export class NewsDataApiClient {
const search = new URLSearchParams({ ...params, apikey: this.#apiKey });
const fullUrl = `${this.#endpointUrl(endpoint)}?${search.toString()}`;
const logUrl = redactApiKey(fullUrl);
const method = ENDPOINT_METHODS[endpoint] ?? 'GET';

for (let attempt = 1; attempt <= this.#maxRetries; attempt += 1) {
this.#log('info', `GET ${logUrl} (attempt ${attempt}/${this.#maxRetries})`);
this.#log('info', `${method} ${logUrl} (attempt ${attempt}/${this.#maxRetries})`);

let res;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), this.#timeout);
try {
res = await this.#fetch(fullUrl, {
method: 'GET',
method,
headers: { Accept: 'application/json' },
signal: controller.signal,
});
Expand Down Expand Up @@ -220,7 +285,7 @@ export class NewsDataApiClient {
throw new NewsdataApiError(`Non-JSON response from API (status ${status})`, status);
}

if (status === 200 && this.#isSuccess(body)) {
if (status === 200 && this.#isSuccess(body, endpoint)) {
if (this.#includeHeaders) {
body.responseHeaders = Object.fromEntries(res.headers.entries());
}
Expand Down Expand Up @@ -325,14 +390,11 @@ export class NewsDataApiClient {
}
}

#isSuccess(body) {
return (
body
&& typeof body === 'object'
&& body.status === 'success'
&& body.results !== null
&& body.results !== undefined
);
#isSuccess(body, endpoint) {
if (!body || typeof body !== 'object' || body.status !== 'success') return false;
// The websocket management endpoints may answer without a `results` field.
if (RESULTS_OPTIONAL.includes(endpoint)) return true;
return body.results !== null && body.results !== undefined;
}

#errorMessage(body, status) {
Expand Down
39 changes: 39 additions & 0 deletions src/core/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,34 @@ export const ENDPOINTS = Object.freeze({
count: 'count',
crypto_count: 'crypto/count',
market_count: 'market/count',
websocket_register: 'websocket/register',
websocket_fetch: 'websocket/fetch',
websocket_delete: 'websocket/delete',
});

// HTTP method per endpoint; anything absent is a GET.
export const ENDPOINT_METHODS = Object.freeze({
websocket_register: 'POST',
websocket_delete: 'DELETE',
});

// The websocket management endpoints answer with a success envelope that may
// carry no `results` field, so they are exempt from the results-present check
// applied to the news endpoints.
export const RESULTS_OPTIONAL = Object.freeze([
'websocket_register', 'websocket_fetch', 'websocket_delete',
]);

// Real-time WebSocket defaults (NewsDataApiWebSocket).
export const WS_BASE_URL = 'wss://ws.newsdata.io/ws/event';
// The feed a registered query matches against.
export const WS_NEWS_TYPE = 'latest';
// Close code the server uses for a permanent rejection.
export const WS_POLICY_VIOLATION = 1008;
export const WS_RECONNECT_DELAY = 1_000; // ms before the first reconnect; doubles each retry
export const WS_RECONNECT_DELAY_MAX = 30_000; // cap on the reconnect delay
export const WS_OPEN_TIMEOUT = 10_000; // ms to wait for the opening handshake

// Endpoints that require both from_date and to_date.
export const REQUIRES_DATE_RANGE = Object.freeze(['count', 'crypto_count', 'market_count']);

Expand Down Expand Up @@ -102,6 +128,19 @@ export const FILTERS = Object.freeze({
'market_id', 'prioritydomain', 'page', 'sentiment', 'removeduplicate', 'size',
'sort', 'tag', 'interval', 'creator', 'datatype', 'sentiment_score',
],
// Real-time query registration. No date/paging filters — a registered query
// matches news as it is published. `news_type` is set by websocketRegister,
// not by the caller.
websocket_register: [
'q', 'qintitle', 'qinmeta', 'country', 'excludecountry', 'category',
'excludecategory', 'language', 'excludelanguage', 'domain', 'domainurl',
'excludedomain', 'prioritydomain', 'timezone', 'full_content', 'image',
'video', 'removeduplicate', 'tag', 'sentiment', 'sentiment_score',
'region', 'organization', 'creator', 'datatype', 'excludefield',
'news_type',
],
websocket_fetch: [],
websocket_delete: ['registration_id'],
});

// Control/meta keys accepted on endpoint methods but not sent as API params.
Expand Down
25 changes: 25 additions & 0 deletions src/core/errors.js
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,31 @@ export class NewsdataServerError extends NewsdataApiError {
}
}

/** A real-time WebSocket stream failure (NewsDataApiWebSocket). */
export class NewsdataWebSocketError extends NewsdataError {
/**
* @param {string} message
* @param {Error|null} [cause] The underlying error.
*/
constructor(message, cause = null) {
super(message);
this.name = 'NewsdataWebSocketError';
if (cause) this.cause = cause;
}
}

/**
* The server rejected the WebSocket connection — bad API key, missing
* WebSocket entitlement, unknown registration_id, device limit reached, or
* exhausted quota. Never retried, regardless of the `reconnect` setting.
*/
export class NewsdataWebSocketAuthError extends NewsdataWebSocketError {
constructor(message, cause = null) {
super(message, cause);
this.name = 'NewsdataWebSocketAuthError';
}
}

/** A network-level failure (DNS, TLS, timeout, abort) prevented the request. */
export class NewsdataNetworkError extends NewsdataError {
/**
Expand Down
Loading
Loading