All Collections
Monsido Starter
Monsido Starter Scan
Monsido Starter Scan

How the Monsido Starter scan works and technical information about the scan process

Updated over a week ago

Section Topics


Introduction

Public website scans are done by both known and unknown crawlers every day. Google, Bing, and other search engines, as well as completely unknown companies, crawl websites to gather information and to monitor performance. What Monsido does is similar, and has no detectable effect on website performance.

The Monsido Starter scan runs automatically every 8 weeks.

A scan report is provided to users that you specify when the subscription starts.


Setup

The Monsido scan is preconfigured and set up to begin with these parameters when the subscription begins:

  • One scan every 8 weeks

  • Scan reports sent automatically after every scan.

    • During the registration phase, you can specify the users that receive these reports.


Scan Progress

To view the progress of a scan:

  1. Click Domain Overview (globe icon) on the page menu bar.

    The location of the Domain Overview button at the top of the landing page.

  2. The Domain Overview page opens.

  3. In the domains list, a message informs that scan is in progress.

    Scan in Progress note on the same row as a domain.

  4. A progress bar, on the same row as a domain with a scan in progress, indicates how much of the website has been scanned so far.

Note: A scan can take several hours. Issues that can affect the scan time are:

  • Domain size

  • Connection speed

  • Website structure.

If there is a message such as:

  • RETRY

  • ON-HOLD

  • Any other questionable status,

Contact your CivicPlus sales representative to be directed to the relevant support team.


Technical information

This section gives technical details about the Monsido Starter Scan.


How the Monsido scan is done

Google Cloud Platform is the hosting environment for Monsido services.

Monsido scans your public-facing websites and does not access or store any confidential data. The only data that Monsido stores (that may not be publicly accessible) are the names and email addresses of our customer's users, which we do not make publicly available.

The Monsido crawler scans data as it finds it. This is normally breadth-first, for example, the scan starts at level 0 (the front page of the website) and proceeds to level 1 (pages that are accessed via links from the front page), and continues in this way.

Monsido scans by using a combination of dynamic discovery using breadth-first approach and XML sitemap if it is explicitly supplied or discovered through robots.txt.


What happens during a Monsido Starter scan

The Monsido crawler first reads every web page on the domain and indexes the content, links, images and files. We do a GET request on every internal HTML page and (if document scanning is enabled) all Word, PowerPoint, and PDF documents. For images, we do a HEAD request, and fall back to a GET request if HEAD request fails.

When the crawl is completed, the collected data is moved from the crawler server to the application server. Depending on the size of your website, this might take some time.

Once the data is moved into place, post-processing begins. Examples of what Monsido does during this stage are:

  1. Classifying every link on the site.

  2. Mapping internal links between pages enabling Monsido to prioritize pages and errors.

  3. Detecting page languages and doing a spellcheck on every page.

  4. Checking for Accessibility issues.

  5. Sending notifications and reports per email.

Monsido Starter sends a scan report via email when all of the above tasks are done.

Note: The Monsido Starter scan does not provide the entire range of data that the full version of the Monsido scan provides.


How the data is stored

All data and backups collected is kept for at least 30 days before being permanently deleted if and when the client exits, with the exception of logs.

Unless otherwise agreed upon, the location of customer data is based on which Monsido branch your contract is with. This means that customer data for USA Monsido offices and subsidiaries will be processed and stored within the USA, customer data for AU Monsido offices and subsidiaries will be processed and stored within AU, and customer data for EU and UK Monsido offices and subsidiaries is processed and stored within the EU.

Login credentials (name, email, one-way hashed password, and data-location ID) are stored centrally in our login service, which resides in the EU.


IP Addresses used by Monsido:

United States & Canada

35.226.117.128

UK & Europe

130.211.65.55

Australia & New Zealand

35.189.0.46


Site performance during a scan

The Monsido scan does not affect site performance.


Did this answer your question?