Page MenuHomePhabricator

jwang (Jennifer)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Jan 13 2020, 11:39 PM (233 w, 11 h)
Availability
Available
LDAP User
Unknown
MediaWiki User
JWang (WMF) [ Global Accounts ]

Recent Activity

Wed, Jun 5

jwang added a comment to T364406: Migrate IP masking dashboard data pipeline to airflow.

DAG files have been submitted to gitlab. A request for review and merge has been made. link

Wed, Jun 5, 8:59 PM · Temporary accounts (Figure out Analytics/Instrumentation for Temp Accounts rollout), Product-Analytics (Kanban)
jwang updated the task description for T364406: Migrate IP masking dashboard data pipeline to airflow.
Wed, Jun 5, 5:32 PM · Temporary accounts (Figure out Analytics/Instrumentation for Temp Accounts rollout), Product-Analytics (Kanban)
jwang added a comment to T364406: Migrate IP masking dashboard data pipeline to airflow.
  • Write updated SQL queries for table creation and updating

Queries have been checked in to Gitlab: link

Wed, Jun 5, 5:31 PM · Temporary accounts (Figure out Analytics/Instrumentation for Temp Accounts rollout), Product-Analytics (Kanban)

Tue, Jun 4

jwang updated the task description for T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.
Tue, Jun 4, 12:44 AM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.

QA code has been cleaned and uploaded to gitlab. gitlab link

Tue, Jun 4, 12:44 AM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

QA code has been cleaned and uploaded to gitlab. gitlab link

Tue, Jun 4, 12:39 AM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
Tue, Jun 4, 12:38 AM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.

QA code has been cleaned and uploaded to gitlab. gitlab link

Tue, Jun 4, 12:37 AM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

May 29 2024

jwang moved T364406: Migrate IP masking dashboard data pipeline to airflow from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
May 29 2024, 5:24 PM · Temporary accounts (Figure out Analytics/Instrumentation for Temp Accounts rollout), Product-Analytics (Kanban)
jwang moved T365143: Analyze usage of dark mode in beta from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
May 29 2024, 5:24 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang added a project to T365143: Analyze usage of dark mode in beta: Product-Analytics (Kanban).
May 29 2024, 5:24 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

May 28 2024

jwang updated the task description for T365143: Analyze usage of dark mode in beta.
May 28 2024, 9:57 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang added a comment to T365143: Analyze usage of dark mode in beta.

Here are the analysis result based on the data collected from beta users in vector-2022 skin on desktop web between may 16, 2024 and may 28 2024. cc @ovasileva.

What percentage of users have changed their color theme?

We don’t have an accurate answer for this question. Here are the data we have and their limitations.

  • 31145 users have enabled beta feature preference cross wikis by May 28, 2024.They would be exposed to the font menu and color theme menu if they visited our website. But we don’t have data on how many of them have visited our website after we deployed the color theme menu in May.
  • The 8066 clicks were made by 1967 unique sessions from May 16 and May 28, 2024. One user could have one or more sessions.
May 28 2024, 9:56 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang updated the task description for T365143: Analyze usage of dark mode in beta.
May 28 2024, 9:39 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang updated the task description for T365143: Analyze usage of dark mode in beta.
May 28 2024, 9:32 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

May 21 2024

jwang added a comment to T364483: Analyze appearance menu usage on pilot wikis.

Here is the summary based on the result collected between May 7, 2024 and May 20, 2024. cc: @ovasileva
Take-aways:

  • The result is similar to the initial result.
  • For logged-in users, after the sudden increase after deployment, the number of clicks fell and stayed flat.
May 21 2024, 5:18 PM · Product-Analytics (Kanban), Web-Team-Backlog

May 13 2024

jwang moved T364483: Analyze appearance menu usage on pilot wikis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
May 13 2024, 10:33 PM · Product-Analytics (Kanban), Web-Team-Backlog
jwang added a comment to T364483: Analyze appearance menu usage on pilot wikis.

Here is the initial summary based on the result collected between May 7, 2024 and May 12, 2024 on pilot wikis.
How many users clicked the font options?

  • Among logged-in users, 2.1% of sessions have changed their text size.
  • Among anonymous users, 1.5% of sessions have changed their text size.
user typeclicksclicked sessionsinitializationinitialized sessionsclick_rate
Logged-in users40311287744885604390.021
Anonymous users27380290304928355258584770.015
  • Daily trend of number of clicks on font options

image.png (962×1 px, 207 KB)

May 13 2024, 10:33 PM · Product-Analytics (Kanban), Web-Team-Backlog

May 10 2024

jwang added a comment to T363238: Create measurement plan and instrumentation spec for IP reputation instrumentation.

I have submitted for LS3C review. Here is the Asana link.

May 10 2024, 5:13 PM · Product-Analytics (Kanban)

May 9 2024

jwang added a project to T364547: Year-end report on FY2023-24 KR WE2.1: Product-Analytics.
May 9 2024, 4:38 PM · Product-Analytics, FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang updated the task description for T364547: Year-end report on FY2023-24 KR WE2.1.
May 9 2024, 4:38 PM · Product-Analytics, FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

May 8 2024

jwang moved T363238: Create measurement plan and instrumentation spec for IP reputation instrumentation from Doing to Needs Review on the Product-Analytics (Kanban) board.
May 8 2024, 6:47 PM · Product-Analytics (Kanban)
jwang added a comment to T363238: Create measurement plan and instrumentation spec for IP reputation instrumentation.

Hi @kostajh, here are the draft of measurement plan and the draft of instrumentation spec. Please review and feel free to edit. Let me know if you think they are ready to submit for legal review.

May 8 2024, 6:47 PM · Product-Analytics (Kanban)
jwang edited projects for T364483: Analyze appearance menu usage on pilot wikis, added: Product-Analytics (Kanban); removed Product-Analytics.
May 8 2024, 6:21 PM · Product-Analytics (Kanban), Web-Team-Backlog

May 7 2024

jwang created T364406: Migrate IP masking dashboard data pipeline to airflow.
May 7 2024, 5:00 PM · Temporary accounts (Figure out Analytics/Instrumentation for Temp Accounts rollout), Product-Analytics (Kanban)

May 6 2024

jwang updated the task description for T359418: Analyze usage of desktop text size beta feature.
May 6 2024, 5:31 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang closed T359418: Analyze usage of desktop text size beta feature, a subtask of T313828: [EPIC] Typography: improve typography and allow for variable typography settings , as Resolved.
May 6 2024, 5:24 PM · Web-Team-Backlog (FY2024-25 Q1 Sprint 2), FY2023-24-WE 2.1 Typography and palette customizations, Epic, Desktop Improvements (Vector 2022)
jwang closed T359418: Analyze usage of desktop text size beta feature, a subtask of T360097: [goal] Deploy reading accessibility settings menu and new typography defaults on desktop , as Resolved.
May 6 2024, 5:24 PM · Web-Team-Backlog (FY2023-24 Q4 Sprint 5), Goal, FY2023-24-WE 2.1 Typography and palette customizations, Epic
jwang closed T359418: Analyze usage of desktop text size beta feature as Resolved.
May 6 2024, 5:24 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang closed T359418: Analyze usage of desktop text size beta feature, a subtask of T360811: Review results of mediawiki_web_ui_actions QA, as Resolved.
May 6 2024, 5:24 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 6)

Apr 30 2024

jwang moved T363238: Create measurement plan and instrumentation spec for IP reputation instrumentation from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Apr 30 2024, 9:24 PM · Product-Analytics (Kanban)
jwang added a comment to T361093: Merge DesktopWeb- and MobileWebUIClickTracking instruments.

@KSarabia-WMF , my understanding is that mediawiki_web_ui_actions will be the merged schema. After all issues in mediawiki_web_ui_actions are resolved, we can move forward to using it as the analysis data source. And DesktopWeb and MobileWebUIClickTracking can be retired.

Apr 30 2024, 4:58 PM · Web-Team-Backlog, Web Team Essential Work 2024

Apr 26 2024

jwang moved T357771: Analyze how many distinct devices edit per day from a given IP address from Doing to Needs Review on the Product-Analytics (Kanban) board.
Apr 26 2024, 8:44 PM · Product-Analytics (Kanban), Temporary accounts

Apr 25 2024

jwang moved T353970: Track metrics on Portuguese Wikipedia relating to IP-editing turn off from Doing to Needs Review on the Product-Analytics (Kanban) board.

The draft is under review now.

Apr 25 2024, 9:32 PM · Product-Analytics (Kanban), Temporary accounts

Apr 23 2024

jwang created T363238: Create measurement plan and instrumentation spec for IP reputation instrumentation.
Apr 23 2024, 9:19 PM · Product-Analytics (Kanban)

Apr 18 2024

jwang updated the task description for T346979: Report on baseline for interface customization.
Apr 18 2024, 9:30 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang moved T361638: Determine number of logged-in editors using each skin on a subset of wikis from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Apr 18 2024, 12:01 AM · Product-Analytics (Kanban), Desktop Improvements (Vector 2022), Web-Team-Backlog
jwang added a project to T361638: Determine number of logged-in editors using each skin on a subset of wikis: Product-Analytics (Kanban).
Apr 18 2024, 12:01 AM · Product-Analytics (Kanban), Desktop Improvements (Vector 2022), Web-Team-Backlog
jwang updated the task description for T361638: Determine number of logged-in editors using each skin on a subset of wikis.
Apr 18 2024, 12:00 AM · Product-Analytics (Kanban), Desktop Improvements (Vector 2022), Web-Team-Backlog

Apr 17 2024

jwang added a comment to T361638: Determine number of logged-in editors using each skin on a subset of wikis.

Both skin preference and global preference reflect the status as of the data collection date, which is April 15, 2024.

Apr 17 2024, 6:30 PM · Product-Analytics (Kanban), Desktop Improvements (Vector 2022), Web-Team-Backlog

Apr 16 2024

jwang moved T362453: Migrate the notebook to fetch data for IP mask dashboard to spark from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Apr 16 2024, 4:30 PM · Product-Analytics (Kanban)

Apr 15 2024

jwang updated the task description for T361638: Determine number of logged-in editors using each skin on a subset of wikis.
Apr 15 2024, 10:00 PM · Product-Analytics (Kanban), Desktop Improvements (Vector 2022), Web-Team-Backlog

Apr 12 2024

jwang updated the task description for T362453: Migrate the notebook to fetch data for IP mask dashboard to spark.
Apr 12 2024, 9:56 PM · Product-Analytics (Kanban)
jwang created T362453: Migrate the notebook to fetch data for IP mask dashboard to spark.
Apr 12 2024, 9:56 PM · Product-Analytics (Kanban)
jwang closed T359993: Slowdown when querying via Hive as Resolved.
Apr 12 2024, 6:29 PM · Data-Platform-SRE (2024.03.25 - 2024.04.14), Data-Platform
jwang added a comment to T359993: Slowdown when querying via Hive.

@JAllemandou @BTullis, Thank you very much for detailed explanation! I will move from hive to presto and spark. I am going to mark this ticket as resolved.

Apr 12 2024, 6:28 PM · Data-Platform-SRE (2024.03.25 - 2024.04.14), Data-Platform

Apr 5 2024

jwang added a comment to T359418: Analyze usage of desktop text size beta feature.

@ovasileva, here are the analysis result. The answer to the third questions is a very rough estimate. Let me know if you disagree with any of the assumptions.

Apr 5 2024, 10:26 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

Apr 3 2024

jwang updated the task description for T359418: Analyze usage of desktop text size beta feature.
Apr 3 2024, 5:04 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

Apr 2 2024

jwang added a comment to T359418: Analyze usage of desktop text size beta feature.

What is the default font value on vector-2022 ? Regular

Apr 2 2024, 9:36 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang triaged T361579: Re-run analysis on which usernames begin with ~2 as High priority.
Apr 2 2024, 4:43 PM · Temporary accounts (Update MediaWiki Core to introduce temp accounts), Product-Analytics (Kanban)
jwang added a project to T361579: Re-run analysis on which usernames begin with ~2: Product-Analytics (Kanban).
Apr 2 2024, 4:43 PM · Temporary accounts (Update MediaWiki Core to introduce temp accounts), Product-Analytics (Kanban)

Mar 29 2024

jwang updated subscribers of T359418: Analyze usage of desktop text size beta feature.

Here is the font size stats on desktop web by skin version. A few questions based on the data

  1. What is the default font value on vector-2022 ?
  2. What does the value disabled stand for on vector-2022?
  3. What is the default font value on vector ? Should we include vector skin in this analysis?
Mar 29 2024, 10:39 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

Mar 26 2024

jwang moved T359418: Analyze usage of desktop text size beta feature from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 26 2024, 6:02 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog
jwang edited projects for T359418: Analyze usage of desktop text size beta feature, added: Product-Analytics (Kanban); removed Product-Analytics.
Mar 26 2024, 6:01 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

Mar 25 2024

jwang updated the task description for T352342: QA WebUIScroll port to the new metrics platform.
Mar 25 2024, 11:16 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

As a followup, I have documented sample rate at data hub.

Mar 25 2024, 11:16 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.
Mar 25 2024, 11:03 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.

As a followup, I have documented the sample rate at datahub. @KSarabia-WMF , please review and confirm whether they are reflecting the current configuration.

Mar 25 2024, 11:02 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
Mar 25 2024, 10:45 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
Mar 25 2024, 10:45 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
Mar 25 2024, 10:44 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.

As a followup, I have documented the current sample rate at https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,event.mobilewebuiactionstracking,PROD)/Documentation?is_lineage_mode=false

Mar 25 2024, 10:43 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Mar 21 2024

jwang added a comment to T357771: Analyze how many distinct devices edit per day from a given IP address.
  • Following the inclusion of client hints in the analysis, there was an average increase of 2 in the maximum number of unique user agents on a daily basis.
    • Throughout January 2024, the daily maximum rose from 6 to 8 unique user agents per IP on English Wikipedia.
    • For some days, the increase in maximum after including client info could be as large as 5.
Mar 21 2024, 12:51 AM · Product-Analytics (Kanban), Temporary accounts

Mar 19 2024

jwang moved T322682: Analyze blocked edit attempts from Triage to Epics on the Product-Analytics board.
Mar 19 2024, 6:30 PM · Trust and Safety Product Team, Product-Analytics
jwang edited projects for T322682: Analyze blocked edit attempts, added: Product-Analytics; removed Product-Analytics (Kanban).
Mar 19 2024, 6:30 PM · Trust and Safety Product Team, Product-Analytics

Mar 13 2024

jwang updated the task description for T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
Mar 13 2024, 9:47 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
Mar 13 2024, 9:46 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
What has been checkedStatusNoteSnapshot of the result from the old schemaSnapshot of the result from the new schema
Pick one session_id, compare the resultPASSCaptured same number of events.
image.png (704×826 px, 129 KB)
image.png (704×558 px, 68 KB)
Pick one pageview_id, compare the resultPASSCaptured same number of events.
image.png (416×598 px, 35 KB)
image.png (402×558 px, 34 KB)
By datePASSThe new schema captured 0.37% more events than the old schema. The new schema captured 0.34% more sessions than the old schema.
image.png (680×1 px, 187 KB)
By actionPASSBetween March 1st and 10th, The new schema captured 0.95% more click events than old schema. The new schema captured 0.36% more init events than old schema. The new schema captured 2.23% more show events than old schema. They are within a 2.5% acceptable variance.
image.png (300×674 px, 38 KB)
image.png (270×764 px, 40 KB)
By event nameBased on the data collected from 2024-03-01 to 2024-03-10: 1) 176 types of events are captured in new schema or old schema. 2) 31 types of events are captured in new schema, but not in old schema. 3) 2 types of events are captured in old schema, but not in new schema. They are menu.preferences and menu.ve-editevent name diff file
By wiki❓Is a difference of 2.6% on commonswiki OK?Between 2024-03-01 and 2024-03-10: New schema captured 819 wikis, while the old schema captured 820. The missed wiki is nycwikimedia. The new schema captured 0.56% more events than the old schema in average. The new schema captured 0.47% more sessions than the old schema in average. The highest different rate of session count is from small wikis.Among the large wikis, the events on commonswiki is 2.6% more in new schema.
image.png (456×1 px, 98 KB)
By skin name❓is it expected that the new schema captured 'vector' and 'vector-2022' skin with agent.client_platform_family='mobile_browser'.Based on the data collected from 2024-03-01 to 2024-03-10: The new schema captured 0.37% more minerva events than old schema. The new schema captured 0.34% more minerva sessions than old schema. 'vector' and 'vector-2022' skins are not captured in old schema, but captured in new schema with agent.client_platform_family='mobile_browser'. To check with engineer whehter it is expected.
image.png (252×708 px, 29 KB)
image.png (286×776 px, 36 KB)
By user typePASSThe difference is within a 2.5% variance.
image.png (232×882 px, 35 KB)
image.png (234×910 px, 36 KB)
agent typePASS
image.png (252×736 px, 32 KB)
image.png (228×734 px, 32 KB)
edit count bucket❓ Is it expected that performer.edit_count_bucket is NULL in new schema for logged out users, while in old schema, event.editCountBucket is '0 edits'.For loggedin users, editcountbucket difference is within 2.5% variance. For loggedout users, in new schema performer.edit_count_bucket is NULL, while in old schema event.editCountBucket is '0 edits'. Need to confirm whether it is expected.
image.png (432×832 px, 60 KB)
image.png (476×1 px, 81 KB)
pageNamespacepage.namespace_id is NULL in new schema
is_dark_mode_on,❓ Is the null in old schema expected?The difference is within a 2.5% variance. The old schema captured some NULLs, while new schema didnot.For the events with null in event.is_dark_mode_on, their kin is also NULL. To check with engineer
image.png (330×1 px, 68 KB)
is_dark_mode_prepared_by_os❓ Is the null in old schema expected?The different is within a 2.5% variance. The old schema captured some NULLs, their skin field is NULL too. To check with engineer
image.png (286×2 px, 73 KB)
dark_mode_setting❓ Is the null in old and new schemas expected?The differences in dark_mode_setting being 0,1, 2, and NULL are within a 2.5% variance.
image.png (342×1 px, 80 KB)
is_full_widthThe difference is within a 2.5% variance. - The old schema captured some NULLs, their skin fields are NULL too . To check with engineer
image.png (264×1 px, 62 KB)
is_media_viewer_enabledFor is_media_viewer_enabled=true, the difference is within a 2.5% variance. For is_media_viewer_enabled=false, the new schema captured 2.55% more events than the old schema. To check with engineer.
image.png (292×2 px, 71 KB)
is_page_preview_onPASSThe difference is within a 2.5% variance
image.png (288×1 px, 71 KB)
is_pinnedPASSThe difference is within a 2.5% variance
image.png (230×722 px, 29 KB)
image.png (248×764 px, 29 KB)
font❓Is font size 0 expected?The differences in font sizes, being small, regular, and large, are within a 2.5% variance. The difference in font size being large exceeds 2.5%. Given the low volume and small absolute difference, we mark it as PASS. Both schemas captured some events where the font size was 0. To check with the engineer.
image.png (464×1 px, 106 KB)
action_context❓ What's the meaning of the field valueneed to document the meanings of the values: stable, stable,amc
image.png (578×784 px, 60 KB)
sample.rate❌ incorrect100% for all wikis and for all type of users
image.png (164×420 px, 14 KB)
is_botperformer.is_bot is NULL in new schema.
image.png (304×672 px, 32 KB)
image.png (188×710 px, 25 KB)
Mar 13 2024, 9:43 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.

Based on the number of events captured in the old and new schema, we believe the new schema is configured with the same sample rate as the old schema, as mentioned in T353029#9621127. However, it is recorded as 100% for all wikis in the new schema.

image.png (142×444 px, 14 KB)

Mar 13 2024, 6:28 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang created T359993: Slowdown when querying via Hive.
Mar 13 2024, 12:46 AM · Data-Platform-SRE (2024.03.25 - 2024.04.14), Data-Platform

Mar 12 2024

jwang moved T357542: QA mobilewebuiactionstracking schema port to the new metrics platform from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 12 2024, 7:36 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a project to T357542: QA mobilewebuiactionstracking schema port to the new metrics platform: Product-Analytics (Kanban).
Mar 12 2024, 7:36 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Mar 11 2024

jwang updated the task description for T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.
Mar 11 2024, 11:10 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.

@KSarabia-WMF , thanks for the info.

Mar 11 2024, 11:09 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.
Mar 11 2024, 11:05 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.
What has been checkedStatusNoteSnapshot of the result from the old schemaSnapshot of the result from the new schema
Pick one session_id, compare the resultPASSCaptured same number of events.
image.png (410×736 px, 61 KB)
image.png (472×582 px, 44 KB)
Pick one pageview_id, compare the resultPASSCaptured same number of events.
image.png (378×632 px, 45 KB)
image.png (468×786 px, 57 KB)
By datePASSThe new schema captured 0.39% more events than the old schema. The new schema captured 0.34% more sessions than the old schema.
image.png (288×524 px, 34 KB)
image.png (284×510 px, 32 KB)
By actionPASSBetween March 1st and Match 5th, the new schema captured 0.18% more click events than old schema. The new schema captured 0.18% more click sessions than old schema. The new schema captured 0.58% more init events than old schema.The new schema captured 0.49% more init sessions than old schema. They are within 2.5% acceptable variance.
image.png (352×780 px, 49 KB)
image.png (334×704 px, 42 KB)
By event name4000+ types of event names in desktopwebuiactionstracking schema schema. Event names contain content info of the pages. . Some event names are in old schema but not in new schema, for example ui.sidebar-toc. Some event names are not in old schema but in new schema, for example, ns=0, most of them are from minerva skineven_name.diff_comparison
By wikiPASSNew schema captured 828 wikis, same as the old schema, in the month of Feb 2024.The highest different rate of session count is from small wikis. The events on nowiktionary is 42.3% fewer in new schema. The difference is reduced to 10% since 2024-02-26. The new schema captured 0.85% more events than the old schema in average.The new schema captured 1.44% more sessions than the old schema in average.
image.png (360×1 px, 51 KB)
By skin name❓ is it expected that the new schema captured 'minerva' skin with agent.client_platform_family='desktop_browser'.Based on the data collected from 20240301 to 20240305. The new schema captured 0.52% more vector events than old schema.The new schema captured 0.50% more vector sessions than old schema. The new schema captured 0.3% more vector2022 events than old schema.The new schema captured 0.25% more vector2022 sessions than old schema. minerva skin is not captured in old schema, but captured in new schema with agent.client_platform_family='desktop_browser'. To check with engineer whehter it is expected.
image.png (222×728 px, 31 KB)
image.png (326×672 px, 38 KB)
By user typePASSBased on the data collected from 20240301 to 20240305. New scheam captured more sessions and events than the old schema, but within 2.5% variance.
image.png (240×874 px, 37 KB)
image.png (240×658 px, 30 KB)
agent typePASS{F42562439}{F42562448}
edit count bucket❓ Is it expected that for logged-out users performer.edit_count_bucket is NULL in new schema, while in old schema, event.editCountBucket is '0 edits'.For logged-in users, editcountbucket difference is within 2.5% variance. For logged-out users, performer.edit_count_bucket is NULL in new schema, while in old schema, event.editCountBucket is '0 edits'. Need to confirm whether it is expected.
image.png (398×770 px, 59 KB)
image.png (460×942 px, 75 KB)
pageNamespacepage.namespace_id is NULL in new schema
image.png (332×800 px, 40 KB)
image.png (170×800 px, 24 KB)
viewportSizeBucketdiff is within 2.5% variance. new schema captured 2620 NULL viewportsizebucket with skin minerva . To check with engineer
image.png (454×806 px, 70 KB)
image.png (514×910 px, 78 KB)
is_dark_mode_on,❓ Is null in old schema expectedThe diff is within 2.5% variance. old schema captured some NULLs, while new schema did not. To check with engineer
image.png (328×968 px, 41 KB)
image.png (226×994 px, 37 KB)
is_dark_mode_prepared_by_os❓ Is null in old schema expectedThe diff is within 2.5% variance. old schema captured some NULLs, while new schema did not. To check with engineer
image.png (310×994 px, 40 KB)
image.png (240×990 px, 37 KB)
dark_mode_setting❓ Is null in old schema expectedThe differences in dark_mode_setting being 0, 2, and NULL are within a 2.5% variance. The difference in dark_mode_setting being 1 is larger than 2.5%. Due to the low volume and small absolute difference, we mark it as a pass.
image.png (330×818 px, 39 KB)
image.png (356×856 px, 41 KB)
is_full_widthThe diff is within a 2.5% variance. The old schema captured some NULLs, while new schema did not.The NULL is from anonymous users. To check with engineer
image.png (328×1 px, 72 KB)
is_media_viewer_enabledPASSThe difference is within a 2.5% variance
image.png (284×1 px, 70 KB)
is_page_preview_onPASSThe difference is within a 2.5% variance
image.png (320×1 px, 70 KB)
is_pinnedPASSThe difference is within a 2.5% variance
image.png (298×1 px, 69 KB)
fontThe diff in font=0,1,2 is within a 2.5% variance.. Some values, like large, null, regular and small, are captured in old schema only. To check with engineer.
image.png (610×1 px, 115 KB)
action_context,❓ is it expectedvalue is desktop for minerva skin in new schema
image.png (350×802 px, 36 KB)
is_botperformer.is_bot is NULL in new schema{F42603014}{F42603033}
sample rateincorrect in new schema
image.png (142×444 px, 14 KB)
Mar 11 2024, 10:54 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Mar 8 2024

jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

@KSarabia-WMF, thanks for checking. Can you also clarify what's the sample rate for logged-in users?

Mar 8 2024, 5:46 AM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Mar 7 2024

jwang moved T353029: QA desktopwebuiactionstracking schema port to the new metrics platform from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 7 2024, 4:52 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Mar 6 2024

jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

Hi, @KSarabia-WMF , Can you confirm if below sample rate captured in the new schema is correct?

Mar 6 2024, 9:46 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T357771: Analyze how many distinct devices edit per day from a given IP address.

@kostajh, please see the findings below.

Methodology

We reviewed the distribution of the number of distinct user agents that appear for a given IP address per day on each pilot wiki candidate and the largest wiki enwiki.
We also reviewed the worst-case scenario: the maximum number of the distinct user agents that appear for a given IP address per day across all wikis.
The analysis is limited to anonymous edits committed between 2024-01-01 and 2024-01-31.

Mar 6 2024, 6:28 PM · Product-Analytics (Kanban), Temporary accounts
jwang added a project to T359418: Analyze usage of desktop text size beta feature: Product-Analytics.
Mar 6 2024, 5:49 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

Mar 5 2024

jwang updated the task description for T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.
Mar 5 2024, 4:50 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.

@KSarabia-WMF, can you also provide the sample rate of the old schema DesktopWebUIActionsTracking? Thanks.

Mar 5 2024, 4:50 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang added a comment to T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.

@KSarabia-WMF, can you also provide the sample rate of the old schema MobileWebUIActionsTracking? Thanks.

Mar 5 2024, 4:49 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T357542: QA mobilewebuiactionstracking schema port to the new metrics platform.
Mar 5 2024, 4:47 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Mar 4 2024

jwang claimed T357771: Analyze how many distinct devices edit per day from a given IP address.
Mar 4 2024, 5:53 PM · Product-Analytics (Kanban), Temporary accounts
jwang moved T357771: Analyze how many distinct devices edit per day from a given IP address from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Mar 4 2024, 5:53 PM · Product-Analytics (Kanban), Temporary accounts

Feb 29 2024

jwang added a comment to T358685: Investigate best metric to measure or proxy reader retention.

HI @ovasileva, please see my investigation summary below.

Feb 29 2024, 1:26 AM · Product-Analytics (Kanban), Web-Team-Backlog

Feb 28 2024

jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

Thanks for checking on it. Regarding 0.2% discrepancy, it can be marked as PASS given 1) it's within variance range , 2.5% variance for daily events across all wikis, that we defined in Metrics Platform Instrument Migration Data QA Process Description ; 2) the new instrumentation is capturing more unique sessions than old instrumentation.

Feb 28 2024, 10:02 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang moved T358685: Investigate best metric to measure or proxy reader retention from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
Feb 28 2024, 8:05 PM · Product-Analytics (Kanban), Web-Team-Backlog
jwang added a project to T358685: Investigate best metric to measure or proxy reader retention: Product-Analytics (Kanban).
Feb 28 2024, 8:05 PM · Product-Analytics (Kanban), Web-Team-Backlog

Feb 26 2024

jwang added a comment to T356335: Update WikimediaEvents "is_dark_mode_on" field.

I'll defer to Jennifer about 2 vs auto. I think it's better to do 2 personally in case these definitions ever change in future this will be more resilient to change.

Feb 26 2024, 6:14 PM · Verified, MW-1.42-notes (1.42.0-wmf.20; 2024-02-27), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog (FY2023-24 Q3 Sprint 3)

Feb 13 2024

jwang added a comment to T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.

Migration of desktopwebuiactionstracking schema is ready for QA.
The mobilewebuiactionstracking schema is pending for migration.

Feb 13 2024, 11:42 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang claimed T353029: QA desktopwebuiactionstracking schema port to the new metrics platform.
Feb 13 2024, 11:40 PM · Patch-For-Review, Web-Team-Backlog (FY2023-24 Q3 Sprint 5), Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Feb 2 2024

jwang added a comment to T356335: Update WikimediaEvents "is_dark_mode_on" field.

Hi, thank you for bringing up and clarifying that.

Feb 2 2024, 7:12 PM · Verified, MW-1.42-notes (1.42.0-wmf.20; 2024-02-27), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog (FY2023-24 Q3 Sprint 3)
jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

@phuedx, Here are some findings from my investigation.

Feb 2 2024, 6:33 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Jan 31 2024

jwang added a comment to T346979: Report on baseline for interface customization.

Here are the baselines for devices with a viewport larger than 1200px. @ovasileva , let me know if you have any questions.

Preview disable rate (viewport > 1200px)

Metric: Number of unique sessions with preview off (non-default)/ total number of unique initialized sessions (viewport > 1200px).
The following statistics are based on the data collected between Dec. 21, 2023 and Dec. 31, 2023

User typeDaily averageStd
Loggedin users44.37%0.27%
Anonymous users3.65%0.12%
Jan 31 2024, 9:23 PM · Product-Analytics (Kanban), FY2023-24-WE 2.1 Typography and palette customizations, Web-Team-Backlog

Jan 25 2024

jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

@phuedx, Thanks for resolving all the questions. I will further investigate the remaining question of why the numbers of events, sessions and pages are slightly higher in the new schema. Will bring it up to you when I have more data.

Jan 25 2024, 8:03 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents
jwang updated the task description for T352342: QA WebUIScroll port to the new metrics platform.
Jan 25 2024, 7:51 PM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents

Jan 20 2024

jwang added a comment to T352342: QA WebUIScroll port to the new metrics platform.

Questions to confirm with engineers

  1. The number of events, sessions and pages are slightly higher in the new schema. Is it expected?
  2. Which field is to capture Spider user agent?
  3. Is access_method captured in agent.client_platform_family in the new schema?
  4. Please review the field mapping table below and confirm whether all entries are as expected.
Field in old schemaField in new schemaValue example
actionactionscroll-to-top
action_contextNULL
action_sourceNULL
action_subtypeNULL
web_session_idperformer.session_ide.g. , '2751f1d9e9a0417cbc1x'
meta.dtmeta.dte.g. "2024-01-16T00:17:25.272Z"
page_idpage.id59519
access_methodagent.client_platform_family❓access_method= 'desktop' ; agent.client_platform_family='desktop_browser'
is_anonperformer.is_logged_intrue, false. The old schema captures the status of being an anoymous user, while the new schema captures the status of being a loggedin users.
skinmediawiki.skinvector-2022
user_agent_map['device_family']MISSING ❓Spider
Jan 20 2024, 1:44 AM · Web-Team-Backlog (FY2023-24 Q3 Sprint 3), Data Products (Data Products Sprint 08), Metrics Platform Backlog, Product-Analytics (Kanban), MediaWiki-extensions-WikimediaEvents