Extract and Link Information

The headless installer installs Analytics Server with Docker and without human interaction. Instead of user prompts, the installer parameters are taken from the properties file.

The installer prompts are defined in the file install-rs-headless.properties:

Start the installation:

./install-rs-headless.sh

The properties file is in the same directory as the script.

Use the --dry-run flag to validate the properties file, print the settings, and exit without changing anything.

Both the headless and interactive installers will update Analytics Server as required. If you have an existing installation of Analytics Server, or you didn't choose to update it on install, you can manually run the scripts to update Analytics Server.

ras_server_1 | raise RosetteException(code, message, url)
ras_server_1 | rosette.api.RosetteException: unauthorizedProfile: Access to profile 
'ad-suggestions' not granted:
ras_server_1 | http://ras_proxy:42/rs/rest/v1/entities

There is not a single one size fits all number here. The best value for max heap size depends on a number of factors:

Please note that it’s not recommended setting the max heap to the amount of physical RAM in the system. More heap doesn’t always translate to better performance, especially depending on your garbage collection settings.

Analytics Server’s data files are loaded into virtual memory. Some endpoints, such as /entities, involve a large amount of data. In order for Analytics Server to operate at its peak performance, we recommend that you reserve enough free memory to allow memory mapping of all our data files so that page misses are minimized at runtime.

To modify the JVM heap for standalone, edit the file server/conf/wrapper.conf and modify the value of wrapper.java.maxmemory.

# Maximum JVM heap in GB
ROSETTE_JVM_MAX_HEAP=32
# Minimum JVM heap in GB
ROSETTE_JVM_MIN_HEAP=32

We also recommend increasing the worker threads to 4, as described in Configuring worker threads for HTTP transport.

The number of models that can be simultaneously trained depends on the size of the models and the memory available.

Once the model is written to disk, it consumes relatively little memory (~2 GB) for runtime requirements. The training and writing phases are much more memory intensive, each consuming approximately three times more memory. Typically, a model actively training will require approximately 10 GB of RAM.

Total memory consumption depends on the number of models being trained simultaneously, as well as the size of the models. The training server is a Java application and all operations use the JVM heap. To allow more simultaneous annotations on more projects, increase the RAM allocated to the JVM in Entity Training Server.

To modify the JVM heap:

The application.yml file controls the configuration of the Event Training Server application. The values in this file rarely change and are relative to the container, meaning the values are only used within the container and have no relevance to the machine running the container.

Server 

This section contains the basic server setup. context-path is the part of the URL prepended to all Event Training Server URLs for example /ets/info. In the container, Event Training Server is listening on port 9999.

server:
  servlet:
    context-path: /ets
  port: 9999

Logging 

This section contains the log setup. The default log level is info and can be changed by setting the ETS_LOGGING_LEVEL value in the .env file. The com.netflix.eureka.cluster is set to ERROR because by default it fills the log with unneeded log messages. The same is true for com.netflix.eureka.registry. If you would like to log everything, the two lines referencing com.netflix.* can be commented out with a #.

logging:
  level:
    ROOT: ${ETS_LOGGING_LEVEL:info}
    com.netflix.eureka.cluster: ERROR
    com.netflix.eureka.registry: WARN

Management 

This section controls the management services, including health and metrics. This service can be run on a different port so the management services are not on the same interface as the Event Training Server API. Note: enabling this management port will require changes to the docker-compose.yml file to expose the port.

The health endpoint was customized to disable the display of disk space reporting, ping reporting and refresh information as it cluttered the response. In addition, the health endpoint is configured to always show details of the dependent services (P-ETS and in training mode, Analytics Server). To change the behavior and get a simple UP/DOWN response set show-details to never.

The following management endpoints are enabled: info, health, metrics, and prometheus. There are approximately 20 additional management endpoints that can be enabled.

Metrics is enabled to expose runtime metric information about the Event Training Server process, memory consumption, threads and CPU usage.

Prometheus is enabled so that Event Training Server can be used as a data source for monitoring applications such as Graphana.

Endpoint timing information is enabled and available using the /ets/prometheus endpoint

management:
# Management can be on a separate port
#  server:
#    port: 9888
  health:
    diskspace:
      enabled: false
    ping:
      enabled: false
    refresh:
      enabled: false
  endpoint:
    health:
      show-details: always
  endpoints:
    web:
      base-path: /
      exposure:
        include: "prometheus, metrics, health, info"
  metrics:
    web:
      server:
        auto-time-requests: true

Eureka 

ETS_HOST is only used when Event Training Server is running remotely to PETS

eureka:
  dashboard:
    path: /eurekadashboard
    enabled: false
  instance:
    appname: JETS
    hostname: ${ETS_HOST:ets-server}
    leaseRenewalIntervalInSeconds: 30
    leaseExpirationDurationInSeconds: 120
    status-page-url: /ets/info
    health-check-url: /ets/health
server:
    enableSelfPreservation: false
  client:
    healthcheck:
      enabled: false
#  As the server we don't want to register with ourselves
    registerWithEureka: false
    fetchRegistry: false
    serviceUrl:
      defaultZone:
 http://${eureka.instance.hostname}:${server.port}/ets/eureka/

Info 

This section determines the Event Training Server operating mode (training or extraction). The ETS_TRUSTSTORE_FILENAME and ETS_KEYSTORE_FILENAME are only defined when running outside a container.

info:
  app:
    name: "events-training-server"
    description: "Rosette Event Extraction and Training Server"
ets:
    operating-mode: "${ETS_MODE:training}"
rs:
  # rsUrl is only required in training configuration
  rsUrl: ${RS_URL:}
  rsConnectTimeoutMS:30000
    ssl:
    enable-outgoing-ssl: ${ENABLE_OUTGOING_SSL:false}
        key-store: ${ETS_KEYSTORE_FILENAME:/certs/keystore.jks}
    key-store-password: ${ETS_KEYSTORE_PW:}
    trust-store: ${ETS_TRUSTSTORE_FILENAME:/certs/truststore.jks}
    trust-store-password: ${ETS_TRUSTSTORE_PW:}
pets:
    minimumVersion: v1.0.0
    connectTimeoutMS: 60000
    readTimeoutMS: 60000
    writeBufferSizeKB: 1000

Springdoc 

springdoc:
  show-actuator: true
  # Enable/disable swagger documentation
  api-docs:
    enabled: true
spring:
  banner:
    location: classpath:ets-banner.txt
  resources:
    add-mappings: false
  cloud:
    discovery:
      client:
        composite-indicator:
          enabled: false
        health-indicator:
          enabled: false

This process describes how to capture the logs for the Event Training Server frontend process, (the J-ETS server). The backend worker (P-ETS) processes logs through the docker subsystem.

Configuring the Log Files

Log File Naming

The default log file name is spring.log. If you prefer a different name, you can change the log file name.

Log Rotation

By default, once logs reach 10 MB they are archived. That is, they are compressed with a date stamp and sequence number such as ets-server.log.2022-03-04.0.gz.The file size at which this occurs can be changed by setting the max-size in the file /basis/ets/config/application.yml.

logging: 
  file:  
    name: /logs/ets-server.log 
    max-size: 20 MB

Supported values for the max-size include MB and KB.

The headless installer installs the indoc coref server without human interaction. Instead of user prompts, the installer parameters are taken from the properties file.

The installer prompts are defined in the file install-coref-headless.properties:

# The directory the Coreference server should be installed in
# Default is /basis/coref
install_dir=/basis/coref

# Load the Coreference server docker image into the docker cache
# Default is true
load_image=true

# The port the Coreference server should listen on
# Default is 5000
coref_port=5000

Start the installation:

./install-coref.sh --headless

To run on the console:

cd /basis/rosette/server/bin
./launch.sh console

To stop in the console:

CTRL-C

To run as a daemon:

./launch.sh start

To stop the daemon:

./launch.sh stop

To run the dockerized version of Analytics Server:

cd /basis/rs/rs-docker 
If running on the console: CTRL-C
If running as a daemon: ./launch.sh stop

To stop the dockerized version:

docker compose down

Analytics Server and Entity Training Server must be running.

Model Training Suite has scripts on each server to monitor the health and status of the system. Run them at startup to verify the system is ready to go.

The scripts are:

where /basis is the default installation directory. If you've installed in a different directory, replace /basis with your install directory.

Each script verifies that the Docker containers have loaded and all components are running.

To check the status of Event Training Server, open a browser and proceed to http:/{host}:{port}/ets/health. The default port is 9999.

The workers indicate that Event Training Server is available and Analytics Server can communicate with it:

{"status": "UP",
"components": {
    "PETS-Workers": {
       "status": "UP",
       "details": {
           "P-ETS Workers": "1 Worker(s) Available"
       }
     },
    "RosetteServer": {
       "status": "UP",
       "details": {
           "Analytics Server": "Available at http://memento:8181/rest/v1"
       }
     }
   }
}

Use the following links to verify the status of each of the servers:

Model Training Suite is shipped with a sample events model that can be used to verify the installation for events. Use the import_project.sh script to import the project.

The superuser can create users and perform all tasks. At install, the superuser is set to admin.

The first time you log in as admin, you will be forced to change the password. The initial login information is:

The superuser password can be reset by running the reset_admin.sh in /basis/ras/scripts. After reset, you will be prompted to change the password when you log in the first time.

./reset_admin.sh '2^$4bu\Qm$4C'

Each component generates logs which may be useful if a problem occurs.

The location of the logs is determined during installation. The files in the default locations are:

The Event Training Server (EvTS) leverages Docker log files, allowing you to customize log file location, rotation, log format (e.g. JSON), and if the logs are stored off the device. To display log information, from the Event Training Server install directory:

docker compose logs 

or to follow the logs:

docker compose logs -f 

See the Docker documentation for configuration and usage information.

Log files are also created during installation. Refer to these files for details on how each component was installed, as well as SSL configuration.

/rest/metrics

The metrics endpoint provides info about the JVM. These values can be used with tools like ELK to graph the overall health of the container. For example, you can graph the percentage of memory used over time to see if you are running out of memory.

curl localhost:8181/rest/metrics

# HELP jvm_buffer_pool_used_bytes Used bytes of a given JVM buffer pool.
 TYPE jvm_buffer_pool_used_bytes gauge
jvm_buffer_pool_used_bytes{pool="mapped",} 1.9689509698E10
jvm_buffer_pool_used_bytes{pool="direct",} 219885.0
jvm_buffer_pool_used_bytes{pool="mapped - 'non-volatile memory'",} 0.0
# HELP jvm_buffer_pool_capacity_bytes Bytes capacity of a given JVM buffer pool.
# TYPE jvm_buffer_pool_capacity_bytes gauge
jvm_buffer_pool_capacity_bytes{pool="mapped",} 1.968950405E10
jvm_buffer_pool_capacity_bytes{pool="direct",} 219885.0
jvm_buffer_pool_capacity_bytes{pool="mapped - 'non-volatile memory'",} 0.0
# HELP jvm_buffer_pool_used_buffers Used buffers of a given JVM buffer pool.
# TYPE jvm_buffer_pool_used_buffers gauge
jvm_buffer_pool_used_buffers{pool="mapped",} 58.0
jvm_buffer_pool_used_buffers{pool="direct",} 17.0```

{"message":"Rosette at your service","time":1467912784915}

{
  "name": "Rosette",
  "version": "1.28.0",
  "buildNumber": "a8ea5010",
  "buildTime": "20231218215507",
  "licenseExpiration": "Perpetual"
}

The /rts/info/server endpoint

$ curl "http://localhost:9080/rts/info/server"

returns the configuration properties along with the version:

{"assetRoot": "/basis/rts",
"concurrentSerializationThreads": 1,
"concurrentTrainThreads": 2,
"concurrentWordclassTrainingThreads": 2,
"memoryCacheTimeout": 180,
"serializationDelay": 300,
"version": "1.0.1",
"workspaceRoot": "/basis/rts/workspaces"}

DAL Connection Test

The DAL connection test verifies the connection between Entity Training Server and the mongo instance on Adaptation Studio by connecting to EnTS directly. If Adaptation Studio is not reachable, this test could hang for 1-2 minutes waiting for the connection.

curl -v -X POST "http://localhost:9080/rts/rex/test-dal-connector" --header 'Content-Type: application/json' -d '{
        "connectorId": "AS_MONGO",
        "corpusType": "ANNOTATED_TRAIN_CORPUS",
        "healthCheck" : "true",
  "config": {
    "projectId": "5f1470b6412ff29b8e4982f3",
    "sampleIds": "5f158b0e412ff29b8e4983b8,5f158b0e412ff29b8e4983b5,5f158b0e412ff29b8e4983b7"
  }
}'

If it is working, it will return:

{"tests":[{"connectorId":"AS_MONGO","message":"Health check passed","success":true}]}

This message indicates that the port is open and mongo is responding to requests.

The DAL connector is configured using the file /basis/rts/config/mongodal_config.yaml.

connectionString: 192.168.0.145:27017
maxSamples: 100000
useSsl: false 
user: root
password: admin
authenticationDB: admin

The connectionString points to the mongo instance in Adaptation Studio which is needed by the DAL connector in Entity Training Server. If the test is not successful, use cURL or a mongo client to test the connection to the mongo server without going through Entity Training Server.

curl 192.168.0.145:27017 

When using cURL, a successful response will be a message from mongo similar to:

"It looks like you are trying to access MongoDB over HTTP on the native driver port." 

This indicates the port is open and mongo is responding to requests.

Ping the server to test that the Entity Training Server is running and confirm that you can connect to it:

curl localhost:9080/rts/info/server

Verify the settings in the .env file in the rs-docker directory

$ cat .env 

returns:

# Connector information
RTS_URL=http://192.168.1.234:9080
RTS_SOURCE_STRING=statistical

Verify that the hostname/port are correct and reachable from the host. In the Analytics Server container, the following command should return 200.

curl $RTS_URL

Ping the server to test that Event Training Server is running and confirm that you can connect to it:

curl -v http://<host>:<port>/rest/v1/events/info

If Analytics Server can't connect to Event Training Server, check the event-extractor-factory-config.yaml file in the rs/config/rosapi directory. Verify that the value of eventTrainingServerUrl is correct and not commented out.

Model Training Suite uses mongo as the backend database to store projects. The mongo client is required to perform a health check of the backend.

For performance, db.enable.FreeMonitoring() can be enabled and will provide an external API with statistics. This is not recommended for use in production.

The Manage Project page in the Studio will display the status of Analytics Server and Entity Training Server.

To run the provided scripts you need a certificate and a key generated for each host that is running a component. Each of the certificates must be signed by the root certificate authority (CA) that will be shared by all hosts running components. All certificates, keys, and root certificate authority files must be stored in PEM format.

Event Training Server, Entity Training Server, and Analytics Server are Java processes, so they require a keystore for the host and truststore for the root CA in a password protected Java Key Store (JKS) format in addition to PEM-format files. There is a script provided, generate-keystores.sh, in the Entity Training Server docker directory that will convert the certificate and key into a password protected JKS file. The JKS file must have a .jks file extension. Additionally, this script will generate a password protected truststore JKS file. For Event Training Server, the generate-keystores.sh script is located in the scripts directory.

As long as the root CA is included, any truststore can be used. The truststore created by the provided script does not have to be used.

Adaptation Studio includes scripts to enable and disable SSL support between the servers. The scripts are found in the following directories:

Enable SSL

Install and test the complete Adaptation Studio installation before enabling SSL. This will verify that everything is properly installed.

For each component:

sudo systemctl restart docker

Disable SSL

Analytics Server is based on the Java-based OSGI with Jetty web server. To enable incoming SSL connections the server configuration must be updated to include the path to the truststore, keystore, and their respective passwords. Additionally, to enable outgoing SSL connections, e.g. Entity Training Server Decoder, custom code etc., the Java virtual machine settings must be updated to include the path to the truststore, keystore, and their respective passwords as well.

The enable-rs-ssl.sh script performs the following actions:

The disable-rs-ssl.sh script reverses the actions above:

A script to create a trust and/or keystore for Analytics Server can be found in both the Entity Training Server and Event Training Server directories. The scripts are identical.

Generating a keystore will require a certificate and key in PEM format.

Generating a truststore will require a Root Certificate Authority (Root CA) certificate in PEM format.

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:http="http://cxf.apache.org/transports/http/configuration"
       xmlns:httpj="http://cxf.apache.org/transports/http-jetty/configuration"
       xmlns:sec="http://cxf.apache.org/configuration/security"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans                 http://www.springframework.org/schema/beans/spring-beans.xsd
        http://cxf.apache.org/transports/http/configuration         http://cxf.apache.org/schemas/configuration/http-conf.xsd
        http://cxf.apache.org/transports/http-jetty/configuration   http://cxf.apache.org/schemas/configuration/http-jetty.xsd
        http://cxf.apache.org/configuration/security                http://cxf.apache.org/schemas/configuration/security.xsd">


    <httpj:engine-factory id="rosette-server-engine-config">
        <httpj:engine port="#{ systemProperties['rosapi.port'] }">
            <httpj:tlsServerParameters>
                <sec:clientAuthentication required="false" />
                <sec:keyManagers keyPassword="[key-pass]">
                    <sec:keyStore type="JKS" password="[keystore-pass]"
                                  file="path/to/keystore.jks"/>
                </sec:keyManagers>
                <sec:trustManagers>
                    <sec:keyStore type="JKS" password="[truststore-pass]"
                                  file="path/to/truststore.jks"/>
                </sec:trustManagers>
            </httpj:tlsServerParameters>
        </httpj:engine>
    </httpj:engine-factory>

</beans>

urlBase=https://0.0.0.0:${rosapi.port}/rest

Entity Training Server is based on the Java-based Open Liberty web container. To enable incoming and outgoing SSL the server configuration must be updated to include the path to the truststore, keystore, and their respective passwords. Additionally the mongodal_config.yaml file (in the config directory) must be updated to set the useSsl flag to true, enabling SSL between Entity Training Server and the mongo instance running on Adaptation Studio.

The enable-rts-ssl.sh script performs the following actions:

The disable-rts-ssl.sh script reverses the actions above:

Adaptation Studio is based on multiple technologies: python server, nginx reverse proxy, mongoDB server and React. As such, the configuration for Adaptation Studio will be different from a Java process. All components internal to Adaptation Studio use nginx for outgoing communication. Mongo is exposed to the host machine and can be protected by SSL. One of the primary differences between the Adaptation Studio scripts and Entity/Event Training Server scripts is that the Adaptation Studio scripts use PEM files rather than JKS files. Also, since Adaptation Studio is made of one incoming connection (mongo) and three outgoing connections (Entity Training Server, Event Training Server, Analytics Server) it is possible to selectively enable SSL for mongodb, Entity Training Server and Analytics Server.

The enable-ras-ssl.sh script performs the following actions:

The disable-ras-ssl.sh script reverses the actions above:

You can enable and disable the SSL connection between the browser and Adaptation Studio.

Event Training Server requires a certificate and key in PEM file format, in addition to a root certificate in PEM format.

The enable-ets-ssl.sh script performs the following actions:

The disable-ets-ssl.sh script performs the following actions:

Indoc coref requires a certificate and key in PEM file format, in addition to a root certificate in PEM format.

The enable-coref-ssl.sh script will prompt you:

Mutual authentication, where both the client and the server verify each other's identities, is the most secure type of authentication. This section describes how to disable Model Training Suite client validation, thus disabling mutual authentication.

This option copies a project created with the NER template to the NER-Rosette Template. The new copy is on the same server.

This option creates a copy of the project on a new server. A different template can be specified.

To restore the entire database from a backup use the restore_mongo.sh script. This restores all projects in the database.

To save and restore an individual project, export and then import the project.

Event extraction takes advantage of the advanced entity extraction capabilities provided by Entity Extractor. Entity Extractor uses pre-trained statistical models to extract the following entity types:

You can also use custom-trained entity extraction models, trained by the Model Training Suite, to extract additional entity types. These models are loaded into Analytics Server. They can be called in the default configuration or through a custom profile.

Entity Extractor also includes rule-based extractors, including statistical regex extractors that can extract additional entity types such as:

The rule-based extractors are not returned by default, To use rule-based extractors, modify the supplementalRegularExpressionPaths in the configuration (rex-factory-config.yaml) file. You can also add custom regex files to create new exact extractors.

When you call the API, add "profileId" = "myProfileId" to the body of the call.

{"content": "The black bear fought the white tiger at London Zoo.",
 "profileId": "group1"
}

New profiles are automatically loaded in Analytics Server. You do not have to bring down or restart the instance to add new models or data to Analytics Server.

When editing an existing profile, you may need to restart Analytics Server. If the profile has been called since Analytics Server was started, the Server must be restarted for the changes to take effect. If the profile has not been called since Analytics Server was started, there is no need to restart.

To add or update models or data, assuming the custom profile root rosette-users and profiles group1 and group2.

The configurations for each endpoint are contained in the factory configuration files. The worker-config.yaml file describes which factory configuration files are used by each endpoint as well as the pipelines for each endpoint. To modify parameter values or any other configuration values, copy the factory configuration file into the profile path and modify the values.

Each profile can include custom data sets. For example, the entities endpoint includes multiple types of data files, including regex and gazetteers. These files can be put into their own directory for entities, known as an overlay directory. This is an additional data directory which takes priority over the default entities data directory.

You can train and deploy a custom model to the entities endpoint for entity extraction. You can either:

In this example, we're going to add the entity types COLORS and ANIMALS to the entities endpoint, using a regex file.

Most endpoints can take either a text block, a file, or a link to a webpage as the input text.  The webpage link is in the form of a URI. To enable passing a URI to an endpoint, the enableDTE flag must be set in the file com.basistech.ws.worker.cfg.

By default, the flag is set to True; URI passing is enabled.

#download and text 
extractorenableDte=true

The limits for the input parameters are in the file /rosapi/constraints.yaml. Modify the values in this file to increase the limits on the maximum input character count and maximum input payload per call.  You can also increase the number of names per list for each call to the name deduplication endpoint.

The default values were determined as optimal during early rounds of performance tests targeting < 2 second response times.  Larger values may cause degradation of system performance. 

To modify the input constraints:

To speed up first call response time, Analytics Server can be pre-warmed by loading data files at startup at the cost of a larger memory footprint.

Most components load their data lazily, meaning that the data required for processing will only be loaded into memory when an actual call hits. This is particularly true for language-specific data. The consequence is that when the very first call with text in a given language arrives at a worker, the worker can take a quite a bit of time loading data before it can process the request.

Pre-warming is Analytics Server's attempt to address the 1st-call penalty by hitting the worker with text in every licensed language it supports at boot time. Then, when an actual customer request comes in, all data will have already been memory mapped and you won't experience a first call delay as the data is loaded. Only languages licensed for your installation will be pre-warmed.

The default is set to false, pre-warm is not enabled.

To set Analytics Server to warm up the worker upon activation

On macOS/Linux or Windows:

With Docker:

Multiple worker threads allow you to implement parallel request processing. Generally, we recommend that the number of threads should be less than the number of physical cores or less than the total number of hyperthreads, if enabled.

You can experiment with 2-4 worker threads per core. More worker threads may improve throughput a bit, but typically won't improve latency. The default value of worker threads is 2.

If the URL for all licensed endpoints are set to local: (not distributed):

If using transport rules in a distributed deployment on macOS/Linux or Windows:

The models trained by the Entity Training Server are statistically-trained models. Multiple statistical models can be deployed and used in each call to the entities endpoint.

curl -s -X POST \
 -H "Content-Type: application/json" \
 -H "Accept: application/json" \
 -H "Cache-Control: no-cache" \
 -d '{"content": "Sample text for extraction"}' \"
http://<PRODSERVER>/rest/v1/entities"

curl -s -X POST \
 -H "Content-Type: application/json" \
 -H "Accept: application/json" \
 -H "Cache-Control: no-cache" \
 -d '{"content": "Sample text for extraction",
 "profileId": "<profileId>"}' \"
http://<PRODSERVER>/rest/v1/entities"

The redactor determines which entity to choose when multiple mentions for the same entity are extracted. The redactor first chooses longer entity mentions over shorter ones. If the length of the mentions are the same, the redactor uses weightings to select an entity mention.

Different processors can extract overlapping entities. For example, a gazetteer extracts "Newton", Massachusetts as a LOCATION, and the statistical processor extracts "Isaac Newton" as a PERSON. When two processors return the same or overlapping entities, the redactor chooses an entity based on the length of the competing entity strings. By default, a conflict between overlapping entities is resolved in favor of the longer candidate, "Isaac Newton".

The redactor can be configured to set weights by:

While events will identify the language automatically, if the language is misidentified, the correct events model will not be used. We recommend you include the language code in your call, where possible.

If no language is provided, and events is unable to auto-detect it, an endpoint may provide a “Language xxx is not supported” error, where xxx indicates the language was not determined.

Either workspaceId or plan can be provided as an option. Both cannot be used in the same call. When using plan, the workspaceId is provided within the plan.

The base event algorithm extracts events when a key phrase and any required role mentions are detected in the document. It does not recognize whether the event happened or didn't happen, also known as the polarity of the event. For example, in a travel event, the following two sentences will both be extracted by the key phrase "travel":

In the example above, "didn't" is an example of a negation cue. The existence of the cue signifies the event is negated.

You can choose to include or ignore negation when you call the events endpoint. The negation option has 4 values:

By default, if you do not pass in a negation parameter, the sentences above return the same event values.

When both, only_positive, or only_negative options are selected, the polarity is included in the response, with the negation cue, if it exists.

The following example had negation set to both in the request.

{
  "events": [
    {
      "eventType": "flight_booking_schema_new_schema.TRAVEL",
      "mentions": [
        {
          "startOffset": 0,
          "endOffset": 23,
          "roles": [
            {
              "startOffset": 0,
              "endOffset": 4,
              "name": "TRAVELER",
              "id": "T0",
              "dataSpan": "John",
              "confidence": 0.90569645,
              "extractorName": "flight_booking_schema_new_schema.per_title",
              "roleType": "flight_booking_schema_new_schema.PER_TITLE"
            },
            {
              "startOffset": 5,
              "endOffset": 13,
              "name": "key",
              "id": "E1",
              "dataSpan": "traveled"
            },
            {
              "startOffset": 17,
              "endOffset": 23,
              "name": "DESTINATION",
              "id": "Q84",
              "dataSpan": "London",
              "confidence": 0.6654963,
              "extractorName": "flight_booking_schema_new_schema.location-entity",
              "roleType": "flight_booking_schema_new_schema.location"
            }
          ],
          "polarity": "Positive",
          "negationCues": []
        }
      ],
      "confidence": 1,
      "workspaceId": "650c4c891c39afa1b071dae3"
    },
    {
      "eventType": "flight_booking_schema_new_schema.TRAVEL",
      "mentions": [
        {
          "startOffset": 25,
          "endOffset": 55,
          "roles": [
            {
              "startOffset": 25,
              "endOffset": 32,
              "name": "TRAVELER",
              "id": "T2",
              "dataSpan": "Charles",
              "confidence": 0.72164702,
              "extractorName": "flight_booking_schema_new_schema.per_title",
              "roleType": "flight_booking_schema_new_schema.PER_TITLE"
            },
            {
              "startOffset": 40,
              "endOffset": 46,
              "name": "key",
              "id": "E2",
              "dataSpan": "travel"
            },
            {
              "startOffset": 50,
              "endOffset": 55,
              "name": "DESTINATION",
              "id": "E3",
              "dataSpan": "Paris",
              "extractorName": "flight_booking_schema_new_schema.location-entity",
              "roleType": "flight_booking_schema_new_schema.location"
            }
          ],
          "polarity": "Negative",
          "negationCues": [
            {
              "startOffset": 33,
              "endOffset": 39,
              "dataSpan": "didn't"
            }
          ]
        }
      ],
      "confidence": 0.89116663,
      "workspaceId": "650c4c891c39afa1b071dae3"
    }
  ]
}

The events endpoint can support event extraction from multiple event models in a single call.

Through the plan options the user can specify a list of event extraction models to be used when extracting event mentions from a document. 

If no workspaceId or plan is specified, then all events models in the instance are used for extraction.

Only models matching the language of the content are called. This can be explicitly set by passing the language code in the call or events will identify the language. Each model is called serially. The response time will increase as additional models are added to the search. It is still faster, however, than making multiple individual calls to each event model.

For each event mention extracted, the response will include the workspaceId of the model which extracted the event mention. Each entity extracted will include the customProfileId (if any) which the extracted entity came from.

{
  "content": "string",
  "language": "string",
  "options": {
    "workspaceId": "string"
  }

{
  "content": "string",
  "language": "string",
}

{
  "content": "string",
  "language": "string",
  "options": {
    "plan": {
       "string": [
         "string"
        ]
    }
  }

{content": "I want flights from Boston to New York",
  "language": "eng",
  "options":
 {  "plan": {
      "eng": ["multi-1", "multi-2","multi-3"
      ]
    }
  }

GET /events/info

GET /events/info?workspaceId={wid}

The event schema defines the event types you are extracting. It includes key phrases, roles, role types, and extractors.

For each key phrase and role, there is a role-type. A role type is made up of one or more extractors. Extractors are reusable components which define the rules and techniques to identify roles and key phrases.

The supported extractor types are:

You cannot modify the schema for a trained model. You can view it through the /events/info endpoint.