tweet_collector

Twitter forms a rich source of information for researchers interested in studying 'the public conversation'. The Academic Research product track is designed to serve the needs of the academic research community. It provides researchers with special levels of access to public Twitter data without any cost. This project is aimed to help researchers to use Academic Research product track to collect tweets of their interest and analyze them.

Table of Contents

• Project Title
  • Table of Contents
  • About the Project
    • Built with
    • License
  • Getting Started
    • Prerequisites
    • Installation
  • Usage
  • Links
  • Contributing
  • Contact
  • Acknowledgements

About the Project

This project provides a set of guidelines + tool, to facilitate the process of collecting tweets and analysing them.

Date: Sep 2021

Researcher:

• Abby Waysdorf (a.s.waysdofr@uu.nl)

Research Software Engineer(s):

• Parisa Zahedi (p.zahedi@uu.nl)
• Roos Voorvaart (r.voorvaart@uu.nl)

Built with

• searchtweets-v2: It is a python package serves as a wrapper for the all Twitter API v2 search endpoints. We use this package to collect tweets.
• elasticsearch/kibana : The collected tweets are visualized using elasticsearch and kibana.
• docker

License

The code in this project is released under MIT License.

Getting Started

The followings are the main steps.

1. Apply for Academic Research product track
2. Setup the environment
3. Search tweets
4. Visualize tweets

Each step is elaborted in details in guideline folder.

Prerequisites

• To install and run this project locally, you need to have the following prerequisites installed.

  • Python 3.8
  • Docker desktop

Installation

To get a local copy up and running follow these simple steps.

1. Clone the repo:

git clone https://github.com/UtrechtUniversity/tweet_collector.git

2. (Optional but recommended) Create and activate a virtual environment

   python -m venv [myenvname]
   source [myenvname]/bin/activate

3. Create package file:

Navigate to tweet_collector folder and run:

poetry build

4. Install package through pip:

pip install ./dist/[tweet_collector_version.whl]

Note: you should run it for the version you have in your dist folder.

Usage

If the installation through pip was followed:

1. Make sure you are inside the virtual environment tweet_collector was installed in
2. Run the tweet_collector command to start collecting the tweets
3. Run the tweet_collector_elastic command to visualise the results in Kibana.

Links

An overview of interesting links related to the project.

• Install Anaconda Python

Contributing

Contributions are what make the open source community an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

To contribute:

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Contact

Parisa Zahedi - (p.zahedi@uu.nl)

Project Link: https://github.com/UtrechtUniversity/tweet_collector

Acknowledgements

• Special thanks to Ton Smeele from Research IT team
• Special thanks to Sytse Groenwold from Research IT team