1{2 "document_id": "lb_qasper_4c9552eec5c238657f8ed5237bf66067d3fdda2409a903b6",3 "family": "public_qasper",4 "split": "test",5 "text": "Introduction\nThe automatic identification, extraction and representation of the information conveyed in texts is a key task nowadays. In fact, this research topic is increasing its relevance with the exponential growth of social networks and the need to have tools that are able to automatically process them BIBREF0.\nSome of the domains where it is more important to be able to perform this kind of action are the juridical and legal ones. Effectively, it is crucial to have the capability to analyse open access text sources, like social nets (Twitter and Facebook, for instance), blogs, online newspapers, and to be able to extract the relevant information and represent it in a knowledge base, allowing posterior inferences and reasoning.\nIn the context of this work, we will present results of the R&D project Agatha, where we developed a pipeline of processes that analyses texts (in Portuguese, Spanish, or English) and is able to populate a specialized ontology BIBREF1 (related to criminal law) for the representation of events, depicted in such texts. Events are represented by objects having associated actions, agents, elements, places and time. After having populated the event ontology, we have an automatic process linking the identified entities to external referents, creating, this way, a linked data knowledge base.\nIt is important to point out that, having the text information represented in an ontology allows us to perform complex queries and inferences, which can detect patterns of typical criminal actions.\nAnother axe of innovation in this research is the development, for the Portuguese language, of a pipeline of Natural Language Processing (NLP) processes, that allows us to fully process sentences and represent their content in an ontology. Although there are several tools for the processing of the Portuguese language, the combination of all these steps in a integrated tool is a new contribution.\nMoreover, we have already explored other related research path, namely author profiling BIBREF2, aggression identification BIBREF3 and hate-speech detection BIBREF4 over social media, plus statute law retrieval and entailment for Japanese BIBREF5.\nThe remainder of this paper is organized as follows: Section SECREF2 describes our proposed architecture together with the Portuguese modules for its computational processing. Section SECREF3 discusses different design options and Section SECREF4 provides our conclusions together with some pointers for future work.\nFramework for Processing Portuguese Text\nThe framework for processing Portuguese texts is depicted in Fig. FIGREF2, which illustrates how relevant pieces of information are extracted from the text. Namely, input files (Portuguese texts) go through a series of modules: part-of-speech tagging, named entity recognition, dependency parsing, semantic role labeling, subject-verb-object triple extraction, and lexicon matching.\nThe main goal of all the modules except lexicon matching is to identify events given in the text. These events are then used to populate an ontology.\nThe lexicon matching, on the other hand, was created to link words that are found in the text source with the data available not only on Eurovoc BIBREF6 thesaurus but also on the EU's terminology database IATE BIBREF7 (see Section SECREF12 for details).\nMost of these modules are deeply related and are detailed in the subsequent subsections.\nFramework for Processing Portuguese Text ::: Part-Of-Speech Tagging\nPart-of-speech tagging happens after language detection. It labels each word with a tag that indicates its syntactic role in the sentence. For instance, a word could be a noun, verb, adjective or adverb (or other syntactic tag). We used Freeling BIBREF8 library to provide the tags. This library resorts to a Hidden Markov Model as described by Brants BIBREF9. The end result is a tag for each word as described by the EAGLES tagset .\nFramework for Processing Portuguese Text ::: Named Entity Recognition\nWe use the named entity recognition module after part-of-speech tagging. This module labels each part of the sentence into different categories such as \"PERSON\", \"LOCATION\", or \"ORGANIZATION\". We also used Freeling to label the named entities and the details of the algorithm are shown in the paper by Carreras et al BIBREF10. Aside from the three aforementioned categories, we also extracted \"DATE/TIME\" and \"CURRENCY\" values by looking at the part-of-speech tags: date/time words have a tag of \"W\", while currencies have \"Zm\".\nFramework for Processing Portuguese Text ::: Dependency Parsing\nDependency parsing involves tagging a word based on different features to indicate if it is dependent on another word. The Freeling library also has dependency parsing models for Portuguese. Since we wanted to build a SRL (Semantic Role Labeling) module on top of the dependency parser and the current released version of Freeling does not have an SRL module for Portuguese, we trained a different Portuguese dependency parsing model that was compatible (in terms of used tags) with the available annotated.\nWe used the dataset from System-T BIBREF11, which has SRL tags, as well as, the other preceding tags. It was necessary to do some pre-processing and tag mapping in order to make it viable to train a Portuguese model.\nWe made 589 tag conversions over 14 different categories. The breakdown of tag conversions per category is given by table TABREF7. These rules can be further seen in the corresponding Github repository BIBREF12. The modified training and development datasets are also available on another Github repositorypage BIBREF13 for further research and comparison purposes.\nFramework for Processing Portuguese Text ::: Semantic Role Labeling\nWe execute the SRL (Semantic Role Labeling) module after obtaining the word dependencies. This module aims at giving a semantic role to a syntactic constituent of a sentence. The semantic role is always in relation to a verb and these roles could either be an actor, object, time, or location, which are then tagged as A0, A1, AM-TMP, AM-LOC, respectively. We trained a model for this module on top of the dependency parser described in the previous subsection using the modified dataset from System-T. The module also needs co-reference resolution to work and, to achieve this, we adapted the Spanish co-reference modules for Portuguese, changing the words that are equivalent (in total, we changed 253 words).\nFramework for Processing Portuguese Text ::: SVO Extraction\nFrom the yield of the SRL (Semantic Role Labeling) module, our framework can distinguish actors, actions, places, time and objects from the sentences. Utilizing this extracted data, we can distinguish subject-verb-object (SVO) triples using the SVO extraction algorithm BIBREF14. The algorithm finds, for each sentence, the verb and the tuples related to that verb using Semantic Role Labeling (subsection SECREF8). After the extraction of SVOs from texts, they are inserted into a specific event ontology (see section SECREF12 for the creation of a knowledge base).\nFramework for Processing Portuguese Text ::: Lexicon Matching\nThe sole purpose of this module is to find important terms and/or concepts from the extracted text. To do this, we use Euvovoc BIBREF6, a multilingual thesaurus that was developed for and by the European Union. The Euvovoc has 21 fields and each field is further divided into a variable number of micro-thesauri. Here, due to the application of this work in the Agatha project (mentioned in Section SECREF1), we use the terms of the criminal law BIBREF15 micro-thesaurus. Further, we classified each term of the criminal law micro-thesaurus into four categories namely, actor, event, place and object. The term classification can be seen in Table TABREF11.\nAfter the classification of these terms, we implemented two different matching algorithms between the extracted words and the criminal law micro-thesaurus terms. The first is an exact string match wherein lowercase equivalents of the words of the input sentences are matched exactly with lower case equivalents of the predefined terms. The second matching algorithm uses Levenshtein distance BIBREF16, allowing some near-matches that are close enough to the target term.\nFramework for Processing Portuguese Text ::: Linked Data: Ontology, Thesaurus and Terminology\nIn the computer science field, an ontology can be defined has:\na formal specification of a conceptualization;\nshared vocabulary and taxonomy which models a domain with the definition of objects and/or concepts and their properties and relations;\nthe representation of entities, ideas, and events, along with their properties and relations, according to a system of categories.\nA knowledge base is one kind of repository typically used to store answers to questions or solutions to problems enabling rapid search, retrieval, and reuse, either by an ontology or directly by those requesting support. For a more detailed description of ontologies and knowledge bases, see for instance BIBREF17.\nFor designing the ontology adequate for our goals, we referred to the Simple Event Model (SEM) BIBREF18 as a baseline model. A pictorial representation of this ontology is given in Figure FIGREF16\nConsidering the criminal law domain case study, we made a few changes to the original SEM ontology. The entities of the model are:\nActor: person involved with event\nPlace: location of the event\nTime: time of the event\nObject: that actor act upon\nOrganization: organization involved with event\nCurrency: money involved with event\nThe proposed ontology was designed in such a manner that it can incorporate information extracted from multiple documents. In this context, suppose that the source of documents is aare a legal police department, where each document isare under the hood of a particular case/crime; furthermoreFurther, a single case can have documents from multiple languages. Now, considering case 1 has 100 documents and case 2 has 100 documents then there is not only a connection among the documents of a single case but rather among all the cases with all the combined 200 documents. In this way, the proposed method is able to produce a detailed and well-connected knowledge base.\nFigure FIGREF23 shows the proposed ontology, which, in our evaluation procedure, was populated with 3121 events entries from 51 documents.\nProtege BIBREF19 tool was used for creating the ontology and GraphDB BIBREF20 for populating & querying the data. GraphDB is an enterprise-ready Semantic Graph Database, compliant with W3C Standards. Semantic Graph Databases (also called RDF triplestores) provide the core infrastructure for solutions where modeling agility, data integration, relationship exploration, and cross-enterprise data publishing and consumption are important. GraphDB has a SPARQL (SQL-like query language) interface for RDF graph databases with the following types:\nSELECT: returns tabular results\nCONSTRUCT: creates a new RDF graph based on query results\nASK: returns \"YES\", if the query has a solution, otherwise \"NO\"\nDESCRIBE: returns RDF data about a resource. This is useful when the RDF data structure in the data source is not known\nINSERT: inserts triples into a graph\nDELETE: deletes triples from a graph\nFurthermore, we have extended the ontology BIBREF21 to connect the extracted terms with Eurovoc criminal law (discussed in subsection SECREF10) and IATE BIBREF7 terms. IATE (Interactive Terminology for Europe) is the EU's general terminology database and its aim is to provide a web-based infrastructure for all EU terminology resources, enhancing the availability and standardization of the information. The extended ontology has a number of sub-classes for Actor, Event, Object and Place classes detailed in Table TABREF30.\nDiscussion\nWe have defined a major design principle for our architecture: it should be modular and not rely on human made rules allowing, as much as possible, its independence from a specific language. In this way, its potential application to another language would be easier, simply by changing the modules or the models of specific modules. In fact, we have explored the use of already existing modules and adopted and integrated several of these tools into our pipeline.\nIt is important to point out that, as far as we know, there is no integrated architecture supporting the full processing pipeline for the Portuguese language. We evaluated several systems like Rembrandt BIBREF22 or LinguaKit: the former only has the initial steps of our proposal (until NER) and the later performed worse than our system.\nThis framework, developed within the context of the Agatha project (described in Section SECREF1) has the full processing pipeline for Portuguese texts: it receives sentences as input and outputs ontological information: a) first performs all NLP typical tasks until semantic role labelling; b) then, it extracts subject-verb-object triples; c) and, then, it performs ontology matching procedures. As a final result, the obtained output is inserted into a specialized ontology.\nWe are aware that each of the architecture modules can, and should, be improved but our main goal was the creation of a full working text processing pipeline for the Portuguese language.\nConclusions and Future Work\nBesides the end–to–end NLP pipeline for the Portuguese language, the other main contributions of this work can be summarize as follows:\nDevelopment of an ontology for the criminal law domain;\nAlignment of the Eurovoc thesaurus and IATE terminology with the ontology created;\nRepresentation of the extracted events from texts in the linked knowledge base defined.\nThe obtained results support our claim that the proposed system can be used as a base tool for information extraction for the Portuguese language. Being composed by several modules, each of them with a high level of complexity, it is certain that our approach can be improved and an overall better performance can be achieved.\nAs future work we intend, not only to continue improving the individual modules, but also plan to extend this work to the:\nautomatic creation of event timelines;\nincorporation in the knowledge base of information obtained from videos or pictures describing scenes relevant to criminal investigations.\nAcknowledgments\nThe authors would like to thank COMPETE 2020, PORTUGAL 2020 Program, the European Union, and ALENTEJO 2020 for supporting this research as part of Agatha Project SI & IDT number 18022 (Intelligent analysis system of open of sources information for surveillance/crime control). The authors would also like to thank LISP - Laboratory of Informatics, Systems and Parallelism.",6 "char_len": 14693,7 "document_hash": "7ff6e3e4886e4448806ed3936af8134256c33cce78512cecb890c2502159622d",8 "questions": [9 {10 "question_id": "q00",11 "qtype": "qasper",12 "question": "Were any of the pipeline components based on deep learning models?"13 }14 ]15}