Overview
Data is essential for informed decision-making and understanding trends in our society.
This guide provides fundamental knowledge to help you better navigate, interpret, and use official statistics responsibly. It explains where data comes from, how to analyse it correctly, how to communicate findings responsibly, and why seasonal adjustments matter.
How are data collected
How data is collected
The Department of Statistics Singapore (DOS) collects data through the following means:
- Surveys such as household expenditure surveys, price surveys, census etc.
- Administrative sources such as births and deaths data from the Immigration and Checkpoints Authority, education data from the Ministry of Education Singapore etc.
Such data is commonly used in various forms of analyses such as to describe or visualise patterns and trends, or to uncover relationship between variables.
DOS also uses alternative data sources and methods such as web-scraping to supplement the information gleaned from traditional surveys. This refers to the collection of data from the internet using the following to search and extract the data required:
- A programming code (i.e., crawler).
- A web tool (i.e., scraper).
An example is the use of online price information in the compilation of the Consumer Price Index (CPI).
Web-scraping principles
Web-scraping allows DOS to gather data from more companies faster and without contacting them directly, making it easier to provide better, more useful statistics.
DOS abides by a set of web-scraping principles to assure website owners that web-scraping is carried out consistently, ethically and transparently:
Abiding by applicable national legislation;
- Minimising burden on the website owners (eg. by adding idle time between requests, or web-scraping at a time of day during which the web server is not expected to be under heavy load)
- Identifying DOS to the website owners when carrying out web-scraping (e.g., in user agent strings).
The data collected may be used by DOS or shared with other public agencies to fulfill public duties, including policy analyses and service delivery.