Frontend

The frontend was developed with React v16 and Redux with Typescript. Similar to the backend, it listens to the WebSocket server for live users to display on the map. Additionally, it will call the Go backend APIs to fetch historical users within a provided time range along with correponding statistics.

The project consists of only a single view (Home.tsx) which serves as a mediator for its multiple child components (e.g. Timeline, SideNav, Socials) by receiving and passing necessary data through props. Consequently, the Home component is also responsible for managing certain state that lives outside of the Redux store using State Hooks. Most of the API calls, performed via Redux actions, are also centralized here.

Notable packages used in the frontend are:

react as the frontend framework
react-redux for Redux bindings in React
@material-ui for React components with Material Design
google-map-react as the Google Maps API wrapper
- Requires setting the Google Maps API key as an environment variable
react-twitter-embed for embedding tweets from the MindFuel account
recharts for charts

For development, it is recommended to use Node v14 and npm v6. For details on versioning of packages, view the package.json file in the frontend project.

Project Structure

├── frontend_react
├── src
│   ├── api             // API and Socket services for connecting to Go backend and WebSocket server
│   ├── components      // Shared React components used on the page (TSX and CSS files)
│   ├── hooks           // Shared custom React hooks
│   ├── res
│   │   └── assets      // Image and vector assets
│   ├── state           // Redux logic
│   │   ├── actions.ts
│   │   ├── hooks.ts
│   │   ├── reducer.ts
│   │   └── store.ts
│   ├── utils           // Helpers and utils (e.g. type definitions) used by multiple components
│   └── views           // Pages of the application
│       └── Home        // The home page of the application
│           ├── Home.tsx
│           └── Home.module.css
├── src
├── .prettierrc.json    // Settings for Prettier
├── .eslintrc.cjs       // Rules for ESLint
└── App.tsx             // Entry point to the application

Linting and Formatting

The frontend uses ESLint and Prettier to maintain consistent code style and formatting. Linting and formatting fixes are automatically run on staged frontend Typescript files through a pre-commit hook using husky. You can also manually run linting to find and fix errors using the following:

$ npm run lint
$ npm run lint-fix

Creating a Production Build

By default, Docker compose is configured to run the development server for the React application via npm start. To instead create the build files to serve for production, run npm run build. Reference the Create React App documentation for more information on creating a production build.

Redux Usage

The project uses Redux architecture primarily for managing the state of live and historical users in addition to more trivial matters like application loading and alert states. For an in-depth overview of Redux, review the documentation on the Redux website. Introducing Redux as it relates to this project, the initial application state seen below is defined and managed in the reducer (reducer.ts). Actions (actions.ts) update this state through various workflows of the application. Note that we use null within the application state to indicate intentional absence of a value.

const initialUserCount: LiveCounts = {
  sessions: 0,
  countries: new Set(),
  cities: new Set(),
};

const initialState: AppState = {
  liveUsers: [],
  historicalUsers: null,
  historicalCounts: null,
  liveCounts: initialUserCount,
  newUser: null,
  loading: false,
  alert: null,
  appUserLocation: null
  heatmapEnabled: false,
  isWebSocketConnected: false,
};

Backend & APIs

The backend for the project is written in Go and performs two functions:

Serves REST APIs for retrieving user information and stats
Acts as a listener to Wonderville's WebSocket and subsequently records new users and updates stats in the MongoDB database.

The socket listener is decoupled from the REST API service through a goroutine, meaning any interruptions to the REST service will not affect the recording of new data from the socket server. The socket listener will also automatically retry connecting every 60 seconds to the WebSocket server upon disconnection. The application logs are outputted to the rest_golang/logs.txt file.

Notable packages used in the backend are:

chi for the REST API service
recws for WebSocket connections
mongo-go-driver for MongoDB connections

Project Structure

├── rest_golang
│   ├── db        // (Package) Retrieving and updating data from MongoDB
│   ├── logger    // (Package) Application logger
│   ├── model     // (Package) Type definitions for API and internal use
│   ├── server    // (Package) REST API service (handler and router)
│   ├── socket    // (Package) WebSocket listener
│   ├── logs.txt  // Application logs
│   └── main.go   // Entry point to the application

APIs

v1/api/users

Methods: GET
Description: Gets historical users and the total counts of users, uniques cities and unique countries relative to the given start date.
Parameters:

Parameter	Required?	Type	Description	Example
`startDate`	Y	String	The start of the date range in ISO string format.	2022-10-04T14:48:00.000Z
`maxUsers`	Y	Integer	The maximum number of users to return.	100
`mapBounds`	Y	Object	The latitude and longitude boundaries used to search for users. Latitude and longitude values are integers.	{ lat: { lower: -49.68, upper: 84.71 }, lng: { lower: -175.25, upper: -52.74 }
`filter`	N	Object	The activity filter value. Valid filter categories are Category or ActivityType. If Category is chosen, you can specify Game, Video, Activity or Story as the filter type.	{ filterCategory: "Category", filterType: "Game" }

Responses
Success: 200 OK
Content:

{
  "users": [
    {
      "type": "wondervilleAsset",
      "payload": {
        "ip": "172.219.38.100",
        "url": "tree-cookies",
        "location": {
          "country_name": "Canada",
          "region_name": "Alberta",
          "city": "Leduc",
          "longitude": -113.5587,
          "latitude": 53.2659
        },
        "asset": {
          "name": "Tree Cookies",
          "url": "tree-cookies",
          "id": 32,
          "uuid": "",
          "type": "Game",
          "imageUrl": "https://wonderville.org/wvAssets/Uploads/Tree-Cookies-Thumb.png",
          "active": true
        },
        "rank": 1
      },
      "date": "2022-12-04T18:03:43.181Z"
    }
  ],
  "counts": {
    "sessions": 1,
    "cities": 1,
    "countries": 1
  }
}

v1/api/activity-stats

Methods: GET
Description: Gets activity hit counts in descending order by the number of hits for the given date range.
Parameters:

Parameter	Required?	Type	Description	Example
`startDate`	N	String	The start of the date range in ISO string format. If this is not included, all-time hit counts will be returned.	2022-10-04T14:48:00.000Z
`endDate`	N	String	The end of the date range in ISO string format. Required if `startDate` is provided.	2022-11-05T14:48:00.000Z
`top`	N	Integer	The top number activities to return.	10

Responses
Success: 200 OK
Content:

{
  "stats": [
    {
      "hits": 81,
      "name": "Save The World",
      "type": "Game",
      "url": "",
      "imageUrl": "https://wonderville.org/wvAssets/Uploads/Save-the-World-Thumb.png",
      "rank": 1
    },
    {
      "hits": 24,
      "name": "Tree Cookies",
      "type": "Game",
      "url": "",
      "imageUrl": "https://wonderville.org/wvAssets/Uploads/Tree-Cookies-Thumb.png",
      "rank": 2
    }
  ]
}

v1/api/activity-filter-options

Methods: GET
Description: Gets a unique list of all activity categories and activity names recorded over time.
Parameters: None
Responses
Success: 200 OK
Content:

{
  "options": [
    {
      "name": "Game",
      "type": "Category"
    },
    {
      "name": "Video",
      "type": "Category"
    },
    {
      "name": "Airborne Experiment",
      "type": "Game"
    },
    {
      "name": "Waste No More",
      "type": "Video"
    }
  ]
}

Database

MongoDB is used as the document database for the project. The wondervilleActivityBoard database holds two collections, users and activityStats.

MongoDB & Docker

The MongoDB server for the project is containerized with Docker and listens on port 27017. The wondervilleActivityBoard database and root user credentials are defined in docker-compose.yml. Docker volumes are used for persisting data from the database and linking the start-up script (mongo-init.js) to the container. The start-up script creates the necessary collections and indexes.

Collections

users

The users collection is used to store messages (i.e. the users) as they are delivered from the Wonderville WebSocket. These messages can be of Wonderville Asset or Wonderville Session type. Wonderville Assets contain an asset key in the payload with details about the game, activity, story or video that the user accessed, whereas Wonderville Sessions only detail the location of the user. Examples of the stored documents for both message types are shown below.

Wonderville Asset:

{
   "type":"wondervilleAsset"
   "date":{
      "$date":"2022-12-04T18:03:43.181Z"  // The date this entry was recorded
   }
   "payload":{
      "ip": "172.219.38.100",
      "url": "tree-cookies",
      "location": {
         "country_name": "Canada",
         "region_name": "Alberta",
         "city": "Leduc",
         "longitude": -113.5586,
         "latitude": 53.2658
      },
      "asset": {
         "name": "Tree Cookies",
         "url": "tree-cookies",
         "id": 32,
         "uuid": "",
         "type": "Game",
         "imageUrl": "https://wonderville.org/wvAssets/Uploads/Tree-Cookies-Thumb.png",
         "active": true
      },
      "rank":1  // The rank as received by the Wonderville Websocket
   }
}

Wonderville Session:

{
   "type": "wondervilleSession",
   "date": {
      "$date": "2022-02-04T09:15:54.386Z"
   },
   "payload":{
      "location":{
         "country_name": "United States",
         "region_name": "Michigan",
         "city": "Monroe",
         "longitude": -83.476,
         "latitude": 41.9053
      }
   }
}

activityStats

The activityStats collections stores information of the all-time hit counts for each game, activity, story or video. Any time a new message is received from the WebSocket, a record is either created or updated. Note that Wonderville Session stats are not included in this collection since they do not have a corresponding asset.

{
   "name": "Solar Energy Defenders",
   "url": "solarenergydefenders",
   "type": "Game",
   "imageUrl": "https://wonderville.org/wvAssets/Uploads/Solar-Energy-Defenders-Thumb.png",
   "hits": 24749
}

Development

This section is geared towards developers who want to contribute or deploy this project.

Docker

The project's three services are containerized with Docker compose. The configuration for the services is specified in docker-compose.yml which uses environment variables from the .env file to set certain parameters. To spin up all three services, run the following command from the root of the project:

$ docker compose --env-file .env up -d

Note: M1 mac users may need to update the docker-compose.yml file to include platform information before the containers can start successfully, for example:

image: arm64v8/mongo:latest
platform: linux/arm64/v8

Environment Variables

It is necessary to set multiple environment variables so that the services can start successfully. This also includes creating and setting the Google Maps Javascript API key which is used by the frontend. An overview of the necessary environment variables is provided below. For convenience during development, these were set in a single .env file in the root of the project, but they can alternatively be split up between each service.

// React
CHOKIDAR_USEPOLLING=true
REACT_APP_GOOGLE_MAPS_API_KEY // The Google Maps Javascript API key
REACT_APP_MINDFUEL_WEBSOCKET  // The WebSocket server address (can either production or mock development server)
REACT_APP_GOLANG_API          // The backend API path

// Go
GO_MINDFUEL_WEBSOCKET         // The WebSocket server address (can either production or mock development server)
GO_ALLOWED_ORIGINS            // The origins which requests are allowed from as a comma separated list, if not set will allow all HTTP and HTTPS requests

// Mongo (used by Docker and Go)
MONGODB_DB_NAME               // The MongoDB database name
MONGODB_USERNAME              // The username for the Mongo DB root user
MONGODB_PWD                   // The password for the Mongo DB root user
MONGODB_URI                   // The Mongo DB connection string (requires the root user credentials)

Updating this documentation

This documentation was built with mdbook. After updating any of the files in the mdbook directory, run mdbook build to create a new documentation build which will be named book. This builds needs to be moved to the top-level of the repository and renamed to docs where it will then be used by either GitHub Actions or GitLab CI/CD to deploy the documentation.

Mock Services

Two services were created to assist with the development process by providing a source of mock data and a development WebSocket server.

mock_data

The mock_data service seeds your MongoDB instance with a sample set of Wonderville asset data (found in the raw_data directory) and randomized stats using the faker and mongo-seeding packages. This is achieved by using JS scripts nested in the data directory with the same name as the collection to populate (e.g. data/users/users.js). Be warned that running the seeder will drop your existing database.

To run the service, ensure the MONGODB_USERNAME, MONGODB_PWD and MONGODB_DB_NAME environment variables are exported and then in the mock_data directory run:

$ npm install
$ node index.js

Your database collections should then automatically be populated with a 4 month range of sample data including 1 month in the future to avoid having to reseed as frequently. The seeding scripts also provide the option of saving the sample data to a file; to do so uncomment the corresponding block of code at the end of the scripts in the data directory.

mock_server

The mock_server service provides a simple insecure WebSocket server which can be used in place of the production Wonderville server during development. Once connected, it will emit a user from the sample_data.json file every 3 seconds. To use the service, set the WebSocket related environment variables in .env (i.e. REACT_APP_MINDFUEL_WEBSOCKET and/or GO_MINDFUEL_WEBSOCKET) to ws://localhost:3210 and restart the Docker containers.

To start the server, in the mock_server directory run:

$ npm install
$ node index.js

The server will start on port 3210.

Wonderville Activity Activity Board Documentation