Sign in
Corpus Search

Identifies keywords, key n-grams, key collocations and terms.

Changed on

Keywords, key n-grams, key collocations and terms are identified by comparing the focus corpus (or a subcorpus) to a refrence corpus (or a subcorpus). It is the equivalent of using the Keywords and terms tool or using the key option in n-grams or the word sketch.

get/search/extract_keywords

Request

  • The document declares no server URL.
  • Auth: HTTP basic

Query parameters

corpnamestring required

Corpus name. To query your own corpus (e.g. username john, corpus mycorpus), use value user/john/mycorpus.

ref_corpnamestring required

Corpus name of the reference corpus, it must have the same processing (the same attributes, the same term grammar).

usesubcorpstring

The name of the subcorpus. The default value empty string refers to the entire corpus. An example for preloaded/bnc2_tt21 can be Written Academic or 1960-1974.

simple_n'1' | '0'

The smoothing parameter for (simple maths) [https://www.sketchengine.eu/documentation/simple-maths/].

wlfilestring

Defines the allow list (formerly known as whitelist), the list of words which should be included in the list. See also wlblacklist.

wlblackliststring

A deny list (formerly known as blacklist) is a list of items that should be excluded from the result. The values should be be separated by a newline symbol (without commas between values). In the URL, the newline symbol is %0A.

attr'lemma' | 'word' | 'TERM' | 'WSCOLLOC'

Switches between the computation of keywords, terms, key n-grams and key collocations. With keywords and n-grams, it also sets the attribute to be used for the computation.

For keywords, set to the required attribute, usually lc, word or lemma.

For n-grams, set to the required attribute and set usengrams, ngrams_n and ngrams_max_n.

For terms, set the attribute to TERM.

For collocations (word sketch triples, equivalent of using the Word Sketch with AS A LIST option in the web interace), set the attribute to WSCOLLOC. Consider using wlpat.

alnum1 | 0

Limits the results to items containing only alphanumeric characters.

onealpha1 | 0

Limits the results to items containing at least one alphanumberic character. Words such as 16-year-old or 3D will be included.

minfreqstring

Sets the minimum frequency of the item.

maxfreqinteger

Sets the maximum frequency of the item.

max_keywordsinteger

The number of items to be returned in the response.

include_nonwords1 | 0

Includes, or excludes, nonwords in the in the result. Nonwords are tokens which do not start with letter of the alphabet (e.g. numbers, punctuation). The regex to match the nonwords is [^[:alpha:]].*. Certain specialized corpora may use their own specific definition of nonwords.

icase1 | 0

Switches to the lc attribute, i.e. the lower-cased version of the corpus to allow case insensitive searching. 1 means that case sensitivity is off, 0 means it is on.

wlpatstring

Sets a regex to filter the results. Relevant only in a simple wordlist.

addfreqsstring

Represent what kind of frequnecy should be calculated. When used with reldocf it is set to docf to calculate document frequency.

reldocf1 | 0

Calculate the document frequency for each item in the result. Must be used with addfreqs set to docf.

usengrams0 | 1

Represent if n-grams should be extracted or just simple keywords.

ngrams_n2 | 3 | 4 | 5

The minimum n-gram length. Usually used with ngrams_max_n and usengrams.

ngrams_max_n2 | 3 | 4 | 5 | 6

The maximum n-gram length. The maximum is 6.

format'json' | 'xml' | 'csv' | 'tsv' | 'txt' | 'xls'

The format of the output. Empty value is interpreted as JSON. Not every endpoint supports all formats.

Response

OK

referece_corpus_namestring
reference_corpus_sizeinteger
reference_subcorpus_sizeinteger
subcorpus_sizeinteger
corpus_sizeinteger
totalinteger
totalfrq1integer
totalfrq2integer
wllimitinteger
notestring
api_versionstring
manatee_versionstring

Example response

{
  "keywords": [
    {
      "item": "galský",
      "score": 2411.16,
      "frq1": 2,
      "frq2": 512,
      "rel_frq1": 4291.8457,
      "rel_frq2": 0.78041,
      "query": "[lemma=\\\"galský\\\"]"
    }
  ],
  "referece_corpus_name": "Slovak Web 2011 (skTenTen11)",
  "reference_corpus_size": 656067998,
  "reference_subcorpus_size": 656067998,
  "subcorpus_size": 466,
  "corpus_size": 466,
  "total": 175,
  "totalfrq1": 466,
  "totalfrq2": 250525622,
  "wllimit": 1000,
  "api_version": "5.63.1",
  "manatee_version": "2.36.7-SkE-2.221",
  "request": {
    "alnum": "1",
    "maxfreq": "0",
    "minfreq": "1",
    "wlpat": ".*",
    "attr": "lemma",
    "keywords": "1",
    "ref_corpname": "preloaded/sktenten11_rft1",
    "simple_n": "1",
    "k_attr": "lemma",
    "include_nonwords": "0",
    "reldocf": ":\"0\"",
    "icase": "1",
    "onealpha": "1",
    "max_keywords": "1000",
    "corpname": "user/matuskostka1/aaaaa_slovak"
  }
}

Changes

  • a2d72b8a93e4Revision changes
    • breaking

      the query request parameter alnum was restricted to a list of enum values

    • breaking

      the query request parameter simple_n was restricted to a list of enum values

    • breaking

      for the query request parameter alnum, default value was changed from 0 to 1

    • breaking

      for the query request parameter simple_n, default value was changed from 1 to 1

    • warning

      deleted the query request parameter k_attr

    • warning

      deleted the query request parameter keywords

    • warning

      deleted the query request parameter oneaplha

    • warning

      deleted the query request parameter terms

    • info

      added the new optional query request parameter addfreqs

    • info

      added the new optional query request parameter ngrams_max_n

    • info

      added the new optional query request parameter onealpha

    • info

      added the new optional query request parameter usengrams

    • info

      added the new enum value 0.00 to the query request parameter alnum

    • info

      added the new enum value 0 to the query request parameter simple_n

    • info

      added the new enum value 1.00 to the query request parameter alnum

    • info

      added the new enum value 1 to the query request parameter simple_n

    • info

      added the new enum value WSCOLLOC to the query request parameter attr

    • info

      for the query request parameter simple_n, the type was generalized from integer to string