Type Alias DataSourceCrawlScope

Scope for th webcrawler

type DataSourceCrawlScope = {
    depth: number;
    downloadFiles?: boolean;
    maxLinks?: number;
    maxSize?: number;
    scope: "all" | "domain" | "subdomain";
}
Index

Properties

depth downloadFiles? maxLinks? maxSize? scope

Properties

depth: number

The number of levels of links the crawler will traverse

downloadFiles?: boolean

if set to true crawler will try to download and index other supported content like pdf:s from links

maxLinks?: number

Maximum number of links per page that will be extracted

maxSize?: number

Optional parameter. Max size of downloaded files in bytes

scope: "all" | "domain" | "subdomain"

The scope that the crawler will crawl.

Settings

Member Visibility

On This Page

Properties