9 Commits

Author SHA1 Message Date
Nicolas
6634d236bf
(feat/fire-1) FIRE-1 (#1462)
* wip

* integrating smart-scrape

* integrate smartscrape into llmExtract

* wip

* smart scrape multiple links

* fixes

* fix

* wip

* it worked!

* wip. there's a bug on the batchExtract TypeError: Converting circular structure to JSON

* wip

* retry model

* retry models

* feat/scrape+json+extract interfaces ready

* vertex -> googleapi

* fix/transformArrayToObject. required params on schema is still a bug

* change model

* o3-mini -> gemini

* Update extractSmartScrape.ts

* sessionId

* sessionId

* Nick: f-0 start

* Update extraction-service-f0.ts

* Update types.ts

* Nick:

* Update queue-worker.ts

* Nick: new interface

* rename analyzeSchemaAndPrompt -> F0

* refactor: rename agent ID to model in types and extract logic

* agent

* id->model

* id->model

* refactor: standardize agent model handling and validation across extraction logic

* livecast agent

* (feat/f1) sdks (#1459)

* feat: add FIRE-1 agent support to Python and JavaScript SDKs

Co-Authored-By: hello@sideguide.dev <hello@sideguide.dev>

* feat: add FIRE-1 agent support to scrape methods in both SDKs

Co-Authored-By: hello@sideguide.dev <hello@sideguide.dev>

* feat: add prompt and sessionId to AgentOptions interface

Co-Authored-By: hello@sideguide.dev <hello@sideguide.dev>

* Update index.ts

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: hello@sideguide.dev <hello@sideguide.dev>
Co-authored-by: Nicolas <nicolascamara29@gmail.com>

* feat(v1): rate limits

* Update types.ts

* Update llmExtract.ts

* add cost tracking

* remove

* Update requests.http

* fix smart scrape cost calc

* log sm cost

* fix counts

* fix

* expose cost tracking

* models fix

* temp: skipLibcheck

* get rid of it

* fix ts

* dont skip lib check

* Update extractSmartScrape.ts

* Update queue-worker.ts

* Update smartScrape.ts

* Update requests.http

* fix(rate-limiter):

* types: fire-1 refine

* bill 150

* fix credits used on crawl

* ban from crawl

* route cost limit warning

* Update generic-ai.ts

* genres

* Update llmExtract.ts

* test server diff

* cletu

---------

Co-authored-by: rafaelmmiller <150964962+rafaelsideguide@users.noreply.github.com>
Co-authored-by: Thomas Kosmas <thomas510111@gmail.com>
Co-authored-by: Ademílson F. Tonato <ademilsonft@outlook.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: hello@sideguide.dev <hello@sideguide.dev>
Co-authored-by: Gergő Móricz <mo.geryy@gmail.com>
2025-04-15 00:19:45 -07:00
Nicolas
f52d6aab91
(feat/deep-research) Improvements to final analysis (#1443)
* Nick: fixes

* Update research-manager.ts

* Update research-manager.ts
2025-04-10 22:59:28 -07:00
Gergő Móricz
6a10f0689d
ACUC: Dynamic Limits (FIR-1641) (#1434)
* extend acuc definition

* kill plan

* stuff

* stupid tests

* feat: better acuc

* feat(acuc): mock ACUC when not using db auth
2025-04-10 18:49:23 +02:00
Nicolas
a50dc106ef
(feat/deep-research) Deep Research Alpha v1 - Structured Outputs + Customizability (#1365)
* Nick:

* Nick: structured output support

* Nick: support for zod and pydantic
2025-03-24 20:13:52 +04:00
Nicolas
e97a279efe
Nick: let user format the analysis (#1351) 2025-03-18 15:51:40 -04:00
Nicolas
289e351c14
(feat/deep-research-alpha) Added Max Urls, Sources and Fixes (#1271)
* Nick: fixes

* Nick:

* Update deep-research-status.ts
2025-02-27 13:24:40 -03:00
Nicolas
5ab86b8b43
(fix/token-slicer) Fixes extract token limit issues (#1236)
* Nick: fixes extract token limit errors

* Update llmExtract.ts

* Update llmExtract.ts
2025-02-21 20:44:42 +01:00
Nicolas
25d9bdb1f6
(feat/ai-sdk) Migrate to AI-SDK (#1220)
* Nick: init

* Update llmExtract.ts

* Update llmExtract.ts

* Nick rename

* fix(v1/types): extract json schema validation

* Update url-processor.ts

* feat(ai-sdk): ollama support

* feat(ai-sdk): further ollama support

* Nick: it is broken btw

* feat(ai-sdk): abstract model adapter

* Update pnpm-lock.yaml

* Update analyzeSchemaAndPrompt.ts

* Nick:

* feat(ai-sdk): ollama support

* doc(SELF_HOST): update with embedding param

* Nick:

* Update ranker.ts

* Nick:

* feat(ai-sdk): fixes

* Update llmExtract.ts

* feat: remove zod-to-json-schema

* fix

* Update llmExtract.ts

* use openai

* fixes

---------

Co-authored-by: Gergő Móricz <mo.geryy@gmail.com>
2025-02-20 22:48:58 +01:00
Nicolas
5c47e97db2
(feat/deep-research) Alpha implementation of deep research (#1202)
* Nick:

* Revert "fix(v1/types): fix extract -> json rename (FIR-1072) (#1195)"

This reverts commit 586a10f40d354a038afc2b67809f20a7a829f8cb.

* Update deep-research-service.ts

* Nick:

* Nick:

* Nick:

* Nick:

* Nick:

* Nick:

* Update deep-research-service.ts

* Nick:

* Update deep-research-service.ts

* Apply suggestions from code review

---------

Co-authored-by: Gergő Móricz <mo.geryy@gmail.com>
2025-02-19 12:44:21 -03:00