Cloudflare has revealed that the most active AI web crawler over the past year is Bytespider, operated by Bytedance, which uses it to gather training data for its AI models, including the ChatGPT rival Doubao. Amazonbot, which indexes content for Alexa, and ClaudeBot, training the Claude chatbot, rank second and third, respectively. OpenAI’s GPTBot comes in fourth place.
Interestingly, while Bytespider leads in requests and blocking frequency, GPTBot ranks second in both areas. Despite this, many website operators remain unaware of these popular AI crawlers visiting their sites.
Cloudflare’s analysis shows that only a small percentage of websites, around 2.98% of the top one million, take measures to block or challenge AI bot requests. The despite the fact that more popular websites are both more frequently targeted by and more likely to block such crawlers.
The study also highlights that although many sites reference GPTBot, CCBot, and Google in their robots.txt files, they do not specifically disallow popular AI crawlers like Bytespider and ClaudeBot. The effectiveness of blocking relies on bot operators respecting these instructions.
The UAE’s AI, Digital Economy and Remote Work Applications Office, alongside the Mastercard Centre for Advanced AI and Cyber Technology in Dubai and First Abu Dhabi Bank (FAB), have unveiled their inaugural ‘AI Challenge’. Aligned with the UAE Strategy for AI, this initiative aims to invigorate the country’s burgeoning AI sector and cultivate opportunities for AI-focused businesses and talent.
Saqr Binghalib, Executive Director of the AI Office in UAE, emphasised the government’s commitment to fostering partnerships with technology leaders and innovators. This collaboration aims to propel technological advancement, shape the future of AI, and reinforce the UAE’s global leadership in emerging fields.
J.K. Khalil, Division President, East Arabia at Mastercard, highlighted AI’s transformative potential across finance, business operations, and global interactions. The AI Challenge underscores Mastercard’s dedication to leveraging AI for positive impact and supporting the UAE’s vision to pioneer innovation in this domain.
The AI Challenge invites seed and Series A startups to propose innovative AI-driven solutions in areas such as cybersecurity, fintech, and productivity enhancement. Finalists will compete for a US$150,000 cash prize, access to Mastercard’s global resources, and enrolment in its acclaimed Start Path programme, aimed at accelerating startup growth through mentoring and global networking opportunities. Interested participants can register online by 25th August 2024 to participate in this groundbreaking initiative.
Residents of Akishima city in western Tokyo are petitioning to block the construction of a large logistics and data centre by Singaporean developer GLP. Over 220 residents have expressed concerns that the centre would harm local wildlife, cause pollution, increase electricity usage, and deplete the city’s groundwater supply.
The group has filed a petition to review the urban planning process that approved GLP’s 3.63-million-megawatt data centre, which is estimated to emit around 1.8 million tons of carbon dioxide annually. They also worry that the project would require cutting down 3,000 of the 4,800 trees on the site, threatening the habitat of Eurasian goshawks and badgers.
The residents are considering arbitration to force GLP to reconsider its plans, with construction set to begin in February and completion expected by early 2029. The opposition comes amidst growing demand for data centres in Japan, where the market is projected to grow significantly over the next few years. GLP has declined to comment on the matter.
Serbia’s Parliament Speaker Ana Brnabic emphasised the significance of the newly unveiled 2024-2030 AI Development Strategy as a pivotal document for the nation. Highlighting its broad impact across sectors such as education, energy, and healthcare, Brnabic underscored AI’s critical role in Serbia’s societal advancement.
Brnabic noted Serbia’s pioneering move in 2019 by adopting its first AI strategy, positioning the country at the forefront of AI development in Southeastern Europe.
She highlighted Serbia’s membership in the Global Partnership for AI, currently chaired by the nation, and announced plans to host a global AI conference in December.
Under the previous strategy, Serbia established the Institute for AI in Novi Sad in 2021 and introduced its first national AI platform. The supercomputer, available free of charge to universities, scientific institutes, and local startups, aims to foster innovation and technological growth in Serbia’s science and technology parks.
The group aims to reduce the barriers to entering the digital economy for low-income populations, particularly in Sub-Saharan Africa and South Asia. The GSMA highlighted that handset affordability is the most significant obstacle preventing people from going online.
In many low and middle-income countries, mobile phones are often the only means of accessing the internet. Currently, 38% of the global population cannot use mobile internet due to high costs and lack of skills. The coalition will work together to improve access to affordable internet-enabled devices, aiming to close the ‘Usage Gap’ that hinders around three billion people from fully participating in the global digital economy.
Hedge funds are increasingly investing in South Korean chipmakers, betting on a surge in demand for high-end memory chips driven by AI advancements and government support. Notable funds, including Britain’s Man Group and Singapore’s FengHe Fund Management, target giants like SK Hynix and Samsung Electronics, which have lagged behind the broader AI sector rally.
FengHe and other investors see SK Hynix as a key player in the AI market, given its significant supply of high-bandwidth memory (HBM) chips to Nvidia. Despite Hynix’s crucial role, its stock trades at a lower multiple than Taiwan’s TSMC, presenting a perceived value opportunity. Additionally, the South Korean government’s 26 trillion won support package for the chip industry and initiatives to enhance shareholder returns add to the appeal of these stocks.
The influx of hedge fund investment has bolstered the stock market in South Korea, with the KOSPI index achieving its best performance in seven months in June. South Korean stocks have attracted the highest inflows among Asian emerging markets this year, with Samsung and Hynix accounting for a significant portion of KOSPI’s market capitalisation. Despite Hynix’s substantial gains, Samsung is expected to catch up in the latter half of the year.
Beyond chipmakers, the AI boom is benefiting other South Korean industries. For instance, HD Hyundai Electric has seen a significant rise in share price, driven by increased power consumption from AI developments. The ongoing US-China technology conflict further ensures demand for South Korean advanced memory chips as Chinese manufacturers struggle under US export restrictions.
South Korean cosmetics giant AmorePacific has seen immense interest in its new AI beauty lab, where robots custom mix face products and advanced technology recommends the most suitable lipstick colours. Customers like Kwon You-jin appreciate the personalised service, which uses AI-generated reports to analyse skin conditions and match products precisely to individual skin tones.
AI technology is becoming increasingly prevalent in the cosmetics industry, with global brands like L’Oréal and Sephora also adopting it to tailor products to customer needs. In 2023, global beauty industry sales, including cosmetics, reached $625.6 billion, showing steady growth since a dip during the COVID-19 pandemic.
AmorePacific employs deep learning and machine learning techniques to recommend the best product choices. The use of AI speeds up product development and reduces human error and variability in consultations. Analysts believe that AI integration will continue accelerating product launches and lowering industry hurdles.
The market for AI in the beauty and cosmetics sector is projected to more than double from $3.27 billion in 2023 to $8.1 billion by 2028. According to Business Research Company, services such as personalised beauty recommendations, skin analysis, diagnostics, and virtual makeup artists are expected to drive this growth.
According to industry estimates, India‘s technology sector will need over 1 million engineers with advanced AI and other tech skills in the next 2-3 years. The demand highlights a considerable skills gap that current education and training systems must address. Sangeeta Gupta of the National Association of Software and Service Companies (NASSCOM) stressed the need for continuous reskilling, as new college graduates can only fill a quarter of these advanced roles.
The sector, employing around 5.4 million people and contributing significantly to India’s GDP, faces a critical challenge in matching workforce skills to job requirements. Major IT firms like Tata Consultancy Services and Larsen and Toubro are already struggling to fill thousands of positions due to this mismatch, which could disadvantage them against global competitors.
Why does this matter?
The issue’s root lies in India’s education system, which needs more practical skill development. NASSCOM predicts the digital talent gap will widen from 25% to 29% by 2028, exacerbating the problem. Prominent economists, including former central bank Governor Raghuram Rajan, warn that poor schooling could hinder the country’s growth prospects, especially with a predominantly young population.
The US National Education Association (NEA) Representative Assembly (RA) delegates have approved the NEA’s first policy statement on the use of AI in education, providing educators with a roadmap for the safe, effective, and accessible use of AI in classrooms.
Since the fall of 2023, a task force of teachers, education support professionals, higher-ed faculty, and other stakeholders has been diligently working on this policy. Their efforts resulted in a 6-page policy statement, which RA delegates reviewed during an open hearing on 24 June and overwhelmingly approved on Thursday.
A central tenet of the new policy is that students and educators must remain at the heart of the educational process. AI should continue the human connection essential for inspiring and guiding students. The policy highlights that while AI can enhance education, it must be used responsibly, focusing on protecting data, ensuring equitable access, and providing opportunities for learning about AI.
The task force identified several opportunities AI presents, such as customising instructional methods for students with disabilities and making classrooms more inclusive. However, they also acknowledged risks, including potential biases due to the lack of diversity among AI developers and the environmental impact of AI technology. It’s crucial to involve traditionally marginalised groups in AI development and policy-making to ensure inclusivity. The policy clarifies that AI shouldn’t be used to make high-stakes decisions like class placements or graduation eligibility.
Why does this matter?
The policy underscores the importance of comprehensive professional learning for educators on AI to ensure its ethical and effective use in teaching. More than 7 in 10 K-12 teachers have never received professional learning on AI. It also raises concerns about exacerbating the digital divide, emphasising that all students should have access to cutting-edge technology and educators skilled in its use across all subjects, not just in computer science.
At the recent World Artificial Intelligence Conference in Shanghai, Chinese GPU developers seized the opportunity to showcase their products in Nvidia’s absence. Prominent companies such as Iluvatar Corex, Moore Threads, Enflame Technology, Sophgo, and Huawei’s Ascend were at the forefront, highlighting their advancements despite significant challenges in manufacturing and software ecosystems.
Enflame Technology emphasised the shift from foreign-dominated computing clusters to a mix of Chinese and foreign GPUs. The company, along with AI solutions firm Infinigence, is promoting compute resources that utilise a variety of chips from both Nvidia and Chinese manufacturers. However, US export restrictions have prevented Nvidia from selling its most advanced chips in China, and several Chinese firms, including Huawei, are struggling with manufacturing hurdles due to being blacklisted by the US.
Huawei’s booth was a major attraction, showcasing its Ascend 910B chips, which train numerous large language models in China. Meanwhile, Enflame presented its Cloudblazer T20 and T21 AI-training chips, benefiting from not being on the US trade blacklist, which allows it access to global foundries like TSMC.
Despite these efforts, Chinese GPUs still need to catch up with their global counterparts regarding performance. Nvidia remains a dominant player, with tailored chips for the Chinese market continuing to be popular. Nvidia is expected to deliver over 1 million H20 GPUs in China this year, generating $12 billion in sales. However, experts highlight that China’s in-house technology still needs to meet its substantial domestic AI demand.
