Sign in
Corpus Search

A list of word frequencies from the specified corpus.

Changed on

This method can be used for generating frequency lists of all tokens, lemmas, word forms etc. or for retrieving frequencies of concrete items. Regex can be used for detailed criteria.

get/search/wordlist

Request

  • The document declares no server URL.
  • Auth: HTTP basic

Query parameters

corpnamestring required

Corpus name. To query your own corpus (e.g. username john, corpus mycorpus), use value user/john/mycorpus.

wlattrstring required
Example:lc

Sets the corpus attribute you want to work with. Some corpora may have specific additional attributes.

Basic examples:

  • word
  • lc
  • lemma
  • lemma_lc
  • lempos
  • lempos_lc
  • tag
  • pos

For list of available word list attributes see /ca/api/corpora/{corpusId}

usesubcorpstring

The name of the subcorpus. The default value empty string refers to the entire corpus. An example for preloaded/bnc2_tt21 can be Written Academic or 1960-1974.

wlnums'frq' | 'docf' | 'arf'

The type of frequency. The values stand for: frq -> absolute or raw frequency, docf -> document frequency. arf -> average reduced frequency.

wlmaxfreqinteger

Sets the maximum frequency limit in the wordlist.

wlminfreqinteger

Sets the minimum frequency limit. Items with a lower frequency will not be included.

wlpatstring

Sets a regex to filter the results. Relevant only in a simple wordlist.

wlsort'frq' | 'docf'

Sets the sorting of the results. The default is frq, i.e. by absolute frequency. Docf means document frequency.

wlblackliststring

A deny list (formerly known as blacklist) is a list of items that should be excluded from the result. The values should be be separated by a newline symbol (without commas between values). In the URL, the newline symbol is %0A.

include_nonwords1 | 0

Includes, or excludes, nonwords in the in the result. Nonwords are tokens which do not start with letter of the alphabet (e.g. numbers, punctuation). The regex to match the nonwords is [^[:alpha:]].*. Certain specialized corpora may use their own specific definition of nonwords.

relfreq1 | 0

Includes the relative frequency of each item in the result.

reldocf1 | 0

Calculate the document frequency for each item in the result. Must be used with addfreqs set to docf.

wlfilestring

Defines the allow list (formerly known as whitelist), the list of words which should be included in the list. See also wlblacklist.

wlicase1 | 0

Sets the case sensitivity of the corpus, i.e. the data are extracted from a lowercased version of the corpus. It is used for case insensitive analysis. Parameter "1" means case sensitivity, "0" means case insensitivity.

wlmaxitemsinteger

Sets the number of items to be returned in the API response. It is not limited for user corpora, in preloaded corpora can be some limitation. This parameter is often used with wlpage to help with pagination in frontend development.

wlpageinteger

To select page of the response. The number of items on the page is specified by parameter wlmaxitems.

format'json' | 'xml' | 'csv' | 'tsv' | 'txt' | 'xls'

The format of the output. Empty value is interpreted as JSON. Not every endpoint supports all formats.

random1 | 0

Parameter that represents if the wordlist is created from the first 10 milions lines of corpus. One if yes, no if he wordlist is created from the whole corpus.

wltype'simple' | 'struct_wordlist'
Example:simple

Parameter to set the format of ouput. Is it always set to simple, for the struct_wordlist is another enpoint called struct_wordlist.

ngrams_n2 | 3 | 4 | 5

The minimum n-gram length. Usually used with ngrams_max_n and usengrams.

ngrams_max_n2 | 3 | 4 | 5 | 6

The maximum n-gram length. The maximum is 6.

nest_ngrams1 | 0

N-grams which are sub-ngrams of a longer n-gram will be grouped together with the longer n-gram. Nesting only works when a ngrams_n and ngrams_max_n are different values.

simple_n'1' | '0'

The smoothing parameter for (simple maths) [https://www.sketchengine.eu/documentation/simple-maths/].

usengrams0 | 1

Represent if n-grams should be extracted or just simple keywords.

Response

OK

new_maxitemsinteger
wllimitinteger

Word list limit, amount of words to be display.

lastpageinteger
notestring

An additional note to displayed results.

totalinteger

Number if displayed items (word - frequency).

totalfrqinteger

Sum of all frequencies.

wlattr_labelstring
frtpstring

Frequency type. Other possible values can be average reduced frequency, document frequency, score.

api_versionstring
manatee_versionstring

Example response

{
  "new_maxitems": 20000,
  "wllimit": 1000,
  "note": "You are allowed to see only 1000 items.",
  "total": 165953,
  "totalfrq": 111680004,
  "items": [
    {
      "str": "the",
      "frq": 6054939,
      "relfreq": 53895.59026
    }
  ],
  "wlattr_label": "word(lowercase)",
  "frtp": "frequency",
  "api_version": "5.62.3",
  "manatee_version": "2.36.7-SkE-2.219.2",
  "request": {
    "wlminfreq": "5",
    "random": "0",
    "include_nonwords": "1",
    "wltype": "simple",
    "wlmaxitems": "20000",
    "wlsort": "frq",
    "wlicase": "1",
    "wlpage": "1",
    "reldocf": "1",
    "wlpat": ".*",
    "relfreq": "1",
    "wlattr": "lc",
    "wlmaxfreq": "0",
    "corpname": "preloaded/bnc2_tt21"
  }
}

Changes

  • a2d72b8a93e4Revision changes
    • breaking

      the query request parameter simple_n was restricted to a list of enum values

    • breaking

      for the query request parameter wlicase, default value 0 was added

    • breaking

      for the query request parameter simple_n, default value was changed from 1 to 1

    • warning

      deleted the query request parameter ngrams_n

    • info

      added the new optional query request parameter usengrams

    • info

      query request parameter random was deprecated

    • info

      added the new enum value 0 to the query request parameter simple_n

    • info

      added the new enum value 1 to the query request parameter simple_n

    • info

      for the query request parameter simple_n, the type was generalized from integer to string