Re: [Analytics] EventLogging Hive Refine currently stalled for some Schemas - Analytics

15 Nov 2018

...
  Does "fixed" mean that the missing data
already been backfilled? I’m seeing gaps (zero events) in Turnilo for Druid-ingested
EL data, for the
timespans between around 6am-16pm on November 13, and 7am-10am on November
12.

Hm.  Fixed means the data has been refined to Hive.  I didn’t check on
Druid EL data.  Marcel, can you check up on that?  This sounds related, we
might need to figure out how to re run the Druid ingestion if/when base
data partitions change.

On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer &lt;tbayer(a)wikimedia.org&gt; wrote:

...
  Hi Andrew,

 PS, offtopic: reminder that data-analysts(a)wikimedia.org has been
 deprecated/retired a while ago; our team can be reached at
 product-analytics(a)wikimedia.org (CCed)

 On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer &lt;tbayer(a)wikimedia.org&gt; wrote:

  Does "fixed" mean that the missing data
already been backfilled? I'm
 seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the
 timespans between around 6am-16pm on November 13, and 7am-10am on November
 12.

 On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto &lt;otto(a)wikimedia.org&gt; wrote:

  OH I’m sorry!  There is a Phab task, and this is
fixed.
 https://phabricator.wikimedia.org/T209407

 Very sorry, I should have updated and linked the task.

 On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc &lt;gilles(a)wikimedia.org&gt;
 wrote:

  Is this issue still ongoing? Is there a
corresponding Phabricator task
 to follow?

 On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto &lt;otto(a)wikimedia.org&gt; wrote:

> Hi all,
>
> Yesterday we upgraded the Hadoop cluster to a newer version.  It seems
> that along the way the job that imports EventLogging data into Hive has
> started failing for some EventLogging datasets.  I’m still investigating,
> but it seems that tables (I believe ones that need to have their schema
> modified during refinement) are failing refinement.  The list of affected
> tables at this moment is large, and will likely only grow as time goes on.
>
> We’re looking into this and will try to have things fixed ASAP.  For
> now you might expect Hive EventLogging data to be delayed.
>
> Sorry for the trouble, and we’ll keep you updated!
> -Andrew Otto
>  Systems Engineer, WMF
>
>
>
> The list of currently affected tables is:
>
> AdvancedSearchRequest
> CentralAuth
> CentralNoticeBannerHistory
> CentralNoticeImpression
> CentralNoticeTiming
> ChangesListFilterGrouping
> ChangesListFilters
> CitationUsage
> CitationUsagePageLoad
> ContentTranslation
> ContentTranslationAbuseFilter
> ContentTranslationCTA
> ContentTranslationError
> ContentTranslationSuggestion
> CpuBenchmark
> EchoInteraction
> EchoMail
> EditAttemptStep
> EditConflict
> EditorActivation
> EUCCVisit
> EventError
> FlowReplies
> GettingStartedRedirectImpression
> GuidedTourButtonClick
> GuidedTourExited
> GuidedTourExternalLinkActivation
> GuidedTourGuiderHidden
> GuidedTourGuiderImpression
> LandingPageImpression
> MediaViewer
> MediaWikiPingback
> MobileAppCategorizationAttempts
> MobileAppUploadAttempts
> MobileWebMainMenuClickTracking
> MobileWebSearch
> MobileWebUIClickTracking
> MobileWikiAppAppearanceSettings
> MobileWikiAppArticleSuggestions
> MobileWikiAppCreateAccount
> MobileWikiAppDailyStats
> MobileWikiAppEdit
> MobileWikiAppFeed
> MobileWikiAppFeedConfigure
> MobileWikiAppFindInPage
> MobileWikiAppInstallReferrer
> MobileWikiAppIntents
> MobileWikiAppiOSFeed
> MobileWikiAppiOSLoginAction
> MobileWikiAppiOSReadingLists
> MobileWikiAppiOSSearch
> MobileWikiAppiOSSessions
> MobileWikiAppiOSSettingAction
> MobileWikiAppiOSUserHistory
> MobileWikiAppLangSelect
> MobileWikiAppLanguageSearching
> MobileWikiAppLanguageSettings
> MobileWikiAppLinkPreview
> MobileWikiAppLogin
> MobileWikiAppMediaGallery
> MobileWikiAppNavMenu
> MobileWikiAppNotificationInteraction
> MobileWikiAppNotificationPreferences
> MobileWikiAppOfflineLibrary
> MobileWikiAppOnboarding
> MobileWikiAppOnThisDay
> MobileWikiAppPageScroll
> MobileWikiAppProtectedEditAttempt
> MobileWikiAppRandomizer
> MobileWikiAppReadingLists
> MobileWikiAppSavedPages
> MobileWikiAppSearch
> MobileWikiAppSessions
> MobileWikiAppShareAFact
> MobileWikiAppStuffHappens
> MobileWikiAppTabs
> MobileWikiAppToCInteraction
> MobileWikiAppWidgets
> MobileWikiAppWiktionaryPopup
> MultimediaViewerAttribution
> MultimediaViewerDimensions
> MultimediaViewerDuration
> MultimediaViewerNetworkPerformance
> NavigationTiming
> PageIssues
> Popups
> PrefUpdate
> Print
> QuickSurveyInitiation
> QuickSurveysResponses
> ReadingDepth
> ResourceTiming
> SaveTiming
> SearchSatisfaction
> SearchSatisfactionErrors
> ServerSideAccountCreation
> TemplateWizard
> TestSearchSatisfaction2
> TranslationRecommendationAPIRequests
> TranslationRecommendationUIRequests
> TranslationRecommendationUserAction
> TwoColConflictConflict
> UniversalLanguageSelector
> UploadWizardErrorFlowEvent
> UploadWizardExceptionFlowEvent
> UploadWizardFlowEvent
> UploadWizardStep
> UploadWizardTutorialActions
> UploadWizardUploadFlowEvent
> VirtualPageView
> WikidataCompletionSearchClicks
> WikimediaBlogVisit
> WikipediaPortal
> WikipediaZeroUsage
> WMDEBannerEvents
> WMDEBannerSizeIssue
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
 _______________________________________________
 Analytics mailing list
 Analytics(a)lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/analytics

 --
 Tilman Bayer
 Senior Analyst
 Wikimedia Foundation
 IRC (Freenode): HaeB

 --
 Tilman Bayer
 Senior Analyst
 Wikimedia Foundation
 IRC (Freenode): HaeB