MBC Group is stepping up efforts to win users for its video on demand (VOD) platform, Shahid. The broadcaster has integrated cutting edge machine learning into the latest edition of its Shahid Plus app that went live in mid-November. Use of ML will be extended to other Shahid users throughout H1 2020.
Shahid now recommends content to users based on their previous viewing behaviour. By adding machine learning (ML) code to Shahid, MBC hopes to unlock the potential of its vast video archive and boost the time viewers spend on the service.
“We have a catalogue of about 100,000 videos – so there is a huge longtail [of content]. But most people are watching the same 1,000 hours or the most popular titles,” said Adriaan Bloem, head of digital infrastructure, MBC Group.
“The question is ‘how to unlock the vast library of content?’. The answer is the ‘suggested for you’, or ‘recommended for you’ [feature] that you see in a lot of VOD apps,” he added.
MBC’s previous efforts to recommend videos to viewers had been based around semi-curated content, combined with simple algorithms based on a viewers’ location. For example, Egyptian males would like a particular type of content.
However, the process was time consuming and ineffective. “It was very manual, with a lot thinking and tweaking and it wasn’t that effective in the end,” said Bloem.
Machine learning now not only automates recommendations, but also personalises them based on previous viewer behaviour, making it (theoretically at least) more accurate. The more somebody uses the app, the better it can suggest content.
“Machine learning doesn’t care where you live, your gender, your age. These things don’t matter anymore because it is looking for similar viewing patterns,” explained Bloem.
Work on the app began by tracking video completion rates and ranking which video content was viewed the most. The raw data was processed multiple times and fed into its ML algorithms. MBC’s development team then started studying, which clusters of viewers had similar watch behaviour. “We can then suggest what people ‘like you’, have watched next,” explained Bloem.
“The process went surprisingly quickly because we did a lot of the leg work before; the data gathering and clean up,” he added.
Since the app went live, MBC has started gathering baseline data on which recommendations are effective. As part of the process to refine the algorithms, the team will run A/B tests that analyse two algorithms to see which performs better against set KPIs.
In the future baseline data can be integrated into dashboards by the broadcaster’s data scientists. The data generated by the ML algorithms can be used to inform the further development of the platform as well as content acquisition.
“We are at least a year off from getting that far,” said Bloem. “But we want to know what the effect is on churn with the users, and whether people spend more time on the app,” he added.
There are also plans to integrate machine learning into MBC’s ad supported-VOD platform (AVOD), simply called Shahid. ML algorithms will build ‘anonymous’ user profiles designed to boost the service’s ‘stickiness’ and keep people watching longer. This will in turn aid the broadcaster to optimise ad inventory and revenue opportunities.
Integration of ML into Shahid is one of the projects to emerge from the broadcaster’ big data programme. Starting in November 2016 and championed by the then CEO Sam Barnett, the broadcaster made a commitment to integrate data science into the decision making process throughout the organisation. Work to build the actual infrastructure started in early 2017.
“We started our big data project three years ago and it was a major investment for the company. Our CEO was onboard because he wants our company to be data driven, which goes against the grain for a creative company,” commented Bloem.
For the last three years MBC has been freeing data from existing analytical tools and siloed databases and connecting it to its central data lake. Rather than link everything to the data lake, the development team has been working closely with the group’s research unit to address clear business cases within the organisation.
The research team takes “large datasets and creates dashboards, where the business can actually go in and slice and dice [the data] in a way that makes sense to them”, explained Bloem.
Each project takes into account the business objective, what data sources are needed to enable that, and what else needs to be done to ‘create value’ out of the data.
“There has to be a data use case that adds value and therefore is worth the investment,” said Bloem. “When we have clear ROI on each [project], we can rationally defend the investment,” he added.
Early projects have leveraged the data lake to measure reach across audiences, analyse effectiveness of commercials and identify the valuable advertising slots.
“There is a huge array of data. It comes from our analytical tools on digital, where we measure the quality of video streams, it comes internally from databases on commercials…,” Bloem explained.
The initial use cases are paving the way for more sophisticated applications. MBC’s research team has conducted internal meetings with the broadcaster’s group directors to showcase the existing dashboards.
“Our COO, Joe [Igoe] has the attitude of let’s start with concreate use cases, start showing people and then the demand should follow — and it has! Research is inundated with requests for analysis,” commented Bloem.
Artificial intelligence and machine learning have the potential to automate an array of tasks for the broadcaster going forward. For instance, AI could enhance archiving of the broadcaster’s video library by enabling it to add a vast amount of metadata to video assets.
“If you can run a video through [an AI] tool and it recognises brands, [or] it understands what is being said and it can add that as metadata to the asset, it might speed up compliance editing,” explained Bloem.
“It can unlock the potential of the archive, it can make it easier to find [content], avoid all sorts of conflicts with sponsors and advertisers. In the next few years this is really going to take off — it is becoming mainstream,” he predicted.
Although MBC is conducting some pilot work on compliance editing, a completed application is still some way off due partly to the subtleties and ‘ever evolving’ nature of the work.
“However, we’re optimistic we can set up tools that will greatly improve the speed of the compliance process by helping the editors identify problematic scenes, starting in 2020 and then improving the toolset over time,” explained Bloem.
There is also potential to use the viewer data from Shahid to assist the content creation process. The idea of AIs writing scripts is some way off, however, data can be used to enhance creative intuition. For example, viewer data might highlight trends or areas for further content investment.
“In any kind of mature process, you have people who work from experience or intuition, or the creative mindset. They have to, almost begrudgingly, start informing that intuition with data,” commented Bloem.
“You should never take away the creative process, even on the business side. But the next stage is that [data] augments the experience of people. It is a much faster way to feed your intuition, rather than just trying this in a vacuum,” he added.
MBC’s evolution into a data-driven broadcaster has meant expanding its own information technology capabilities. Amazon Web Services (AWS) has been an important partner since MBC Group shifted its server and data infrastructure into a serverless cloud environment. The web services giant has also provided a prototype team that has played a key role in the broadcaster’s move into ML.
“If you want to get to a high level of machine learning there is a lot of development work. It is pretty hard-core tech. Amazon has the cutting edge — the latest tools that are probably years ahead of other providers,” commented Bloem.
MBC also brought in CGI from Finland, which pitched in with big data and ML expertise.
Adding to the international flavour of the project were AWS developers from Turkey based in London, a solutions architect in Spain and MBC’s own tech office in Jordan.
Despite partnering with international firms to start its ML project, the broadcaster is keen to build its in-house machine learning capabilities. Consequently, it has been expanding its Jordan operation and eventually aims to phase out external ML resources.
“We want to build our machine learning team in our tech office in the coming year,” added Bloem.
However, finding people with the right ML skills and the necessary business knowledge has proven near-enough impossible. “Those people are probably working in Silicon Valley and earning half a million a year,” admitted Bloem.
As result, MBC Group has focused on hiring talented local computer science staff and providing them with an opportunity to learn on the job. MBC Group “has to attract talent, retain talent and give them the opportunity to learn”, from the senior team, he said.
Although Jordan has a good crop of computer science graduates from the local universities, the group is still working with outsourced partners in Vietnam to meet its technology and business objectives.
As MBC continues to improve its data platform and AI and ML skillset, the requirement for developers will escalate as the technology is incorporated in both customer-facing and backend operations. “Machine learning is going in multiple directions — there is the big data angle to it, both internally and [with] user facing functionality. [ML] will also start to inform us on transforming the platform and improving over the years,” Bloem concluded.