Logo Felix Elsner

Event Extraction using NLP with Geocoding

Stumbled upon a few interesting papers regarding automated extraction of data from text:

Near-Real Time Analysis of War and Economic Activity during Russia’s Invasion of Ukraine (2022-2023)

This paper explains the VIINA project and applies observations, strangely enough, to agriculture. It contains luminosity analysis (from satellite imagery).

The author also does an analysis of his own data against ACLED, ICEWS and GDELT databases of conflict information.

VIINA / Violent Incident Information from News Articles

VIINA is focused on the 2022 Russian Invasion of Ukraine. It ingests (local) news as well as liveuamap using Natural Language Processing (NLP) for events (strikes, civilian harm, territory control) and provides geocoding as well as categorizes each event with predicted probabilities it belongs to each category.

Geolocating Political Events in Text (2019)

Introducing Mordecai 3: A Neural Geoparser and Event Geocoder in Python

Synthetically generated text for supervised text analysis (2023)

Ukraine War – Russian Missile Data Points

For the purposes of research, I have compiled different estimates of the usage of Precision Guided Munitions (PGM) and other missile types, mostly by the Ukrainian MoD but also other sources.

Please see the dedicated project page: Russian PGM data.

At the dedicated Ukraine War project page, further data about different missile usage assessments (ACLED, Texty.org.ua), about the usage of loitering munitions (such as ZALA Lancet, KUB), day-to-day developments of territorial control and network disruptions can be found.

Interested researchers are welcome to reach me at my contact email for further discussion.

Academic Sanctions Databases

This is a comprehensive overview of sanctions databases and how to acquire them, aimed at researchers. Covered datasets: HSE (1985, 1990, 2009), TIES (2009, 2014), TSC (2018), GSDB (2020, 2021, 2022), EUSANCT (2022), IST (2022), GIGA (2012).

HSE (1985, 1990, 2009)

“HSE” stands for the initials of Hufbauer, Schott and Elliott (later “HSEO” with Oegg included).

Authors: Gary Clyde Hufbauer, Jeffrey J. Schott, Kimberly Ann Elliott and Barbara Oegg.

Releases: 1985 (1st edition), 1990 (2nd edition), 2009 (3rd edition). For an overview of all editions, see Google Books.

Book: Economic Sanctions Reconsidered, 3rd Edition, Paper and Economic Sanctions Reconsidered, 3rd edition, plus CD-ROM.

BibTeX:

@book{hufbauer_economic_1985,
  address = {Washington, D.C. : Cambridge, Mass},
  title = {Economic sanctions reconsidered: history and current policy},
  isbn = {978-0-88132-017-6 978-0-262-08139-9},
  shorttitle = {Economic sanctions reconsidered},
  publisher = {Institute for International Economics ; Distributed by MIT Press},
  author = {Hufbauer, Gary Clyde and Schott, Jeffrey J. and Elliott, Kimberly Ann},
  year = {1985},
}

Introduction chapter available as PDF; selected case studies available as PDF.

Dataset files: Not available on the internet, only via purchase of the CD-ROM.

Code: Code for deriving the conclusions does not seem to be available.

TIES (2009, 2014)

“TIES” stands for The Threat and Imposition of Economic Sanctions.

Authors: T. Clifton Morgan, Navin Bapat, and Valentin Krustev.

Releases: 2009 (covering 1971-200), 2014 (covering 1945-2005).

Papers: “The Threat and Imposition of Economic Sanctions, 1971–2000”, available from JSTOR; “Threat and imposition of economic sanctions 1945–2005: Updating the TIES dataset”, available from SAGEPub.

BibTeX (original paper:

@article{morgan_threat_2009,
  title = {The {Threat} and {Imposition} of {Economic} {Sanctions}, 19712000},
  volume = {26},
  issn = {0738-8942},
  url = {https://www.jstor.org/stable/26275122},
  number = {1},
  journal = {Conflict Management and Peace Science},
  author = {Morgan, T. Clifton and Bapat, Navin and Krustev, Valentin},
  year = {2009},
  note = {Publisher: Sage Publications, Ltd.},
  pages = {92--110},
}

BibTeX (updated paper:

@article{morgan_threat_2014,
  title = {Threat and imposition of economic sanctions 19452005: {Updating} the {TIES} dataset},
  volume = {31},
  issn = {0738-8942},
  shorttitle = {Threat and imposition of economic sanctions 19452005},
  url = {https://doi.org/10.1177/0738894213520379},
  doi = {10.1177/0738894213520379},
  language = {en},
  number = {5},
  journal = {Conflict Management and Peace Science},
  author = {Morgan, T. Clifton and Bapat, Navin and Kobayashi, Yoshiharu},
  month = nov,
  year = {2014},
  note = {Publisher: SAGE Publications Ltd},
  pages = {541--558},
}

Home page: Threat and Imposition of Economic Sanctions (TIES) Data at The University of North Carolina at Chapel Hill (UNC).

Dataset files: Available from the home page as TIES Version 4 (also 3.5) Excel files with a user manual (PDF).

Code: Code for deriving the conclusions does not seem to be available.

TSC (2018)

“TSC” stands for the UN Targeted Sanctions Consortium Database.

Authors: Thomas J Biersteker, Sue E Eckert, Marcos Tourinho, and Zuzana Hudáková.

Releases: 2018.

Paper: “UN targeted sanctions datasets (1991–2013)”, available via SAGEPub.

BibTeX:

@article{biersteker_targeted_2018,
  title = {{UN} targeted sanctions datasets (19912013)},
  volume = {55},
  issn = {0022-3433},
  url = {https://doi.org/10.1177/0022343317752539},
  doi = {10.1177/0022343317752539},
  language = {en},
  number = {3},
  journal = {Journal of Peace Research},
  author = {Biersteker, Thomas J and Eckert, Sue E and Tourinho, Marcos and Hudáková, Zuzana},
  month = may,
  year = {2018},
  note = {Publisher: SAGE Publications Ltd},
  pages = {404--412},
}

Home page: Targeted Sanctions Initiative at the Graduate Institute, Geneva.

Dataset files: Scroll down on the home page, where you will find Excel and SPSS formats, the database codebook (PDF) and the UN Targeted Sanctions qualitative database (PDF.

Code: Code for deriving the conclusions does not seem to be available.

Addendum: Mr. Biersteker has developed the TSC into the UN Sanctions App, available on the Web and for mobile devices.

GSDB (2020, 2021, 2022)

“GSDB” stands for The Global Sanctions Data Base.

Authors: Gabriel Felbermayr, Erdal Yalcin, Yoto Yotov, Aleksandra Kirilakha, and Constantinos Syropoulos.

Releases: 2020, 2021, 2022.

Papers:

BibTeX (2020):

@article{felbermayr_global_2020,
  title = {The {Global} {Sanctions} {Data} {Base}},
  volume = {forthcoming},
  doi = {10.1016/j.euroecorev.2020.103561},
  journal = {European Economic Review},
  author = {Felbermayr, Gabriel and Yalcin, Erdal and Yotov, Yoto and Kirilakha, Aleksandra and Syropoulos, Constantinos},
  month = may,
  year = {2020},
}

BibTeX (2021 update):

@techreport{kirilakha_global_2021,
  type = {School of {Economics} {Working} {Paper} {Series}},
  title = {The {Global} {Sanctions} {Data} {Base}: {An} {Update} that {Includes} the {Years} of the {Trump} {Presidency}},
  url = {https://ideas.repec.org/p/ris/drxlwp/2021_010.html},
  number = {2021-10},
  institution = {LeBow College of Business, Drexel University},
  author = {Kirilakha, Aleksandra and Felbermayr, Gabriel and Syropoulos, Constantinos and Yalcin, Erdal and Yotov, Yoto},
  month = feb,
  year = {2021},
}

BibTeX (2022 update):

@misc{syropoulos_global_2022,
  address = {Rochester, NY},
  type = {{SSRN} {Scholarly} {Paper}},
  title = {The {Global} {Sanctions} {Data} {Base} – {Release} 3: {COVID}-19, {Russia}, and {Multilateral} {Sanctions}},
  shorttitle = {The {Global} {Sanctions} {Data} {Base} – {Release} 3},
  url = {https://papers.ssrn.com/abstract=4291735},
  doi = {10.2139/ssrn.4291735},
  language = {en},
  author = {Syropoulos, Constantinos and Felbermayr, Gabriel J. and Kirilakha, Aleksandra and Yalcin, Erdal and Yotov, Yoto V.},
  year = {2022},
}

Home page: globalsanctionsdatabase.com and project page at the Kiel Institute for the World Economy.

Dataset files: Not directly available to download, but writing to the authors via the listed contact email gets a swift response within a few days.

The authors say this about the data:

The GSDB is freely available and we will be happy to share it with interested researchers, who can request it by e-mail at (address withheld).

In order to ensure the consistency of the GSDB data, we only make the GSDB data available from this webpage. Distribution of the GSDB data through other websites or institutions is not permitted. We kindly ask all users not to upload the data independently to another platform.

Code: Code for deriving the conclusions does not seem to be available.

EUSANCT (2022)

Authors: Patrick M Weber and Gerald Schneider.

Releases: 2022.

Paper: “Post-Cold War Sanctioning by the EU, the UN, and the US: Introducing the EUSANCT Dataset”, available via SAGEPub.

BibTeX:

@article{weber_post-cold_2022,
  title = {Post-{Cold} {War} sanctioning by the {EU}, the {UN}, and the {US}: {Introducing} the {EUSANCT} {Dataset}},
  volume = {39},
  issn = {0738-8942, 1549-9219},
  shorttitle = {Post-{Cold} {War} sanctioning by the {EU}, the {UN}, and the {US}},
  url = {http://journals.sagepub.com/doi/10.1177/0738894220948729},
  doi = {10.1177/0738894220948729},
  language = {en},
  number = {1},
  journal = {Conflict Management and Peace Science},
  author = {Weber, Patrick M and Schneider, Gerald},
  month = jan,
  year = {2022},
  pages = {97--114},
}

Home page: EUSANCT: Does Supranational Coercion Work? Onset, Impact and Effectiveness of EU Sanctions at the University of Konstanz.

Dataset files: On the home page, both the case-level and the dyadic datasets with their respective codebooks are available as Excel and (Stata) .dta files. The case summaries are available as PDF.

Code: Code for deriving the conclusions does not seem to be available.

IST (2022)

“IST” stands for the International Sanctions Termination Dataset.

Authors: Hana Attia, Julia Grauvogel, and Christian von Soest.

Releases: 2022.

Papers:

  • “The Termination of International Sanctions: Explaining Target Compliance and Sender Capitulation”, available from ResearchGate or ScienceDirect.
  • “How Do International Sanctions End? Towards a Process-Oriented, Relational, and Signalling Perspective”, available from JSTOR and from GIGA as a working paper (look for the Download GIGA WP 320 button under Ms. Attia’s name).
  • “International Sanctions Termination, 1990–2018: Introducing the IST dataset”, available from SAGEPub (Open Access) or ResearchGate.

BibTeX (European Economic Review):

@article{article,
  author = {Attia, Hana and Grauvogel, Julia and von Soest, Christian},
  year = {2020},
  month = {08},
  pages = {103565},
  title = {The Termination of International Sanctions: Explaining Target Compliance and Sender Capitulation},
  volume = {129},
  journal = {European Economic Review},
  doi = {10.1016/j.euroecorev.2020.103565}
}

BibTeX (2020 GIGA working paper):

@techreport{10.2307/resrep21217,
  URL = {http://www.jstor.org/stable/resrep21217},
  author = {Julia Grauvogel and Hana Attia},
  institution = {German Institute of Global and Area Studies (GIGA)},
  title = {How Do International Sanctions End?: Towards a Process-Oriented, Relational, and Signalling Perspective},
  year = {2020}
}

BibTeX (2022 Journal of Peace Research article):

@article{attia_international_2022,
  title = {International {Sanctions} {Termination}, 19902018: {Introducing} the {IST} dataset},
  issn = {0022-3433},
  shorttitle = {International {Sanctions} {Termination}, 19902018},
  url = {https://doi.org/10.1177/00223433221087080},
  doi = {10.1177/00223433221087080},
  language = {en},
  journal = {Journal of Peace Research},
  author = {Attia, Hana and Grauvogel, Julia},
  month = aug,
  year = {2022},
  note = {Publisher: SAGE Publications Ltd},
  pages = {00223433221087080},
}

Home page: The Termination of International Sanctions: Causes, Processes and Domestic Consequences at the German Institute for Global and Area Studies (GIGA) at the Leibniz-Institut für Globale und Regionale Studien.

Dataset files: The Excel spreadsheet and codebook as PDF are available from Harvard Dataverse, as linked to from here.

Code: Code for deriving the conclusions does not seem to be available.

GIGA (2012)

“GIGA” stands for German Institute for Global and Area Studies, the research body at the Leibniz-Institut für Globale und Regionale Studien that the authors are affiliated with.

Authors: Christian von Soest and Clara Portela.

Releases: 2012.

Various publications associated with this dataset.

BibTeX (as suggested by search.gesis.org):

@misc{SDN-10.7802-1346,
  author = "Portela, Clara and von Soest, Christian",
  title = "GIGA Sanctions Dataset",
  year = "2012",
  howpublished = "German Institute for Global and Area Studies (GIGA). Datenfile Version 1.0.0, https://doi.org/10.7802/1346",
  doi = "10.7802/1346",
}

Home page: GIGA Sanctions Dataset at the Leibniz-Institut für Globale und Regionale Studien; as part of wider research effort Ineffective Sanctions? External Sanctions and the Persistence of Autocratic Regimes.

Dataset files: From the GESIS archives, the Excel spreadsheet and the codebook are available (after checking the box agreeing to the terms of service).

Code: Code for deriving the conclusions does not seem to be available.

Supplement

Comparion of HSE, TIES and TSC datasets, as compiled by an article for TSC: Available from ResearchGate.

Ukraine War – Russia: Sanctions busting and trade sources

This collection contains a large number of reports and sources about sanctions evasion schemes, components of weapons systems that shouldn’t be there and some background information about the Russian military-industrial complex and enforcement activities.

Analyses of components inside weapons systems fielded by the RuAF

Information about Russian military-industrial complex

Sanctions busting

Ukraine War – Missile strikes data

Possible sources:

  • ACLED Ukraine Conflict Monitor - takes data from (amongst others, but primarily) liveuamap. Datasets require (free) account, only divides between shellings (and similar) and air strikes (and similar), which runs risk of being too coarse a granularity
  • liveuamap - large dataset, API access costly, quite unfiltered
  • Civilian harm dashboards
  • Geolocation-focused efforts
  • Attack map by Texty.org.ua - useful, with lots of links for sources, but also lumps in similar types of attacks so utility somewhat diminished
  • Telegram
  • Ukrainian MoD (via Telegram, Facebook, ZSU website], CinC AFU, KPS ZSU (Air Force, Telegram) - Hard to parse manually since their reporting structure changed regularly. Mostly unverifiable claims without publicly visible proof, numbers don’t add up over time
  • Second-hand reports via Ukrinform, UP, Kyiv Independent and other news outlets which mostly report on statements by Ukrainian MoD
  • Russian “LPR”/”DPR” sources - Through their certain degree of freedom, sometimes quite accurate and honest assessments, if you dig through the mud

Discarded sources:

  • Map of Destruction by “Anti-Corruption headquarters”, in cooperation with Kyiv School of Economics - Has lately (as of 2023-01) pivoted to showing reconstruction progress, which is nice to see, but diminishes utility as data source for the specific avenue of missile strikes
  • Russian government-aligned sources - MoD Russia, rybar (both wildly inaccurate)

Ukraine War – Analyzing Russian LMUR strikes

During the war in Ukraine, the Russian Armed Forces have utilized a variety of Precision Guided Munitions (PGM) requiring high-tech components. One such weapon is the Izdelie 305 (also known as “LMUR”) guided missile, produced by Rostec subsidiary KB Mashinostroyeniya.

This analysis will solely utilize data from visually confirmed usages in the Ukraine war compiled by the “Oryx Spioenkop” authors. The article is called Tracking The Threat: Russia’s Izdeliye 305 (LMUR).

Taking each instance of a strike and extracting the date when a recording of the usage was (re-)posted to social media, one can plot the number of strikes each month. Because there exists a delay between when the strike actually occurred and the date of publishing the footage, a constant of 3 days is subtracted from post dates.

LMUR strikes

The first published strikes occurred in June of 2022 and the data stretches to October of the same year. Published strikes peaked in July 2022 and fell to their lowest in September 2022. With a reported price of 14.2 million roubles, or $227,000, as well as limited stockpiles, Russian forces might be hesitant or simply unable to sustain usage of this comparatively advanced anti-armor weapon as their failed invasion drags on.

Caveats apply; the Russian MoD might have decided to reduce publishing strike footage while actually ramping up strikes and the Oryx authors might not have compiled all available footage (or have some still pending review).

A cursory search of Twitter suggests newer posts of LMUR strikes, but these might in reality be reposts of old footage. Further research is certainly welcome.

Page 1 of 7

Next