Initial Performance Tests Show Snapdragon X Elite Lagging Behind iPhone 12 - Book4 Edge's Surprise Reviews

Copilot+ Showcases Cutting-Edge Snapdragon Notebooks: Join Our Live Testing Event

Refresh

June 18, 2024 at 10:29 PM

Slow Setup Begins

So we’re in the middle of setting up our Surface Laptop 15-inch with Snapdragon Elite X CPU and it is taking . . . forever. As part of the OOBE (Out of the Box Experience), Windows 11 demanded to run updates, which it claims will take 30 minutes.

During the update process, we’ve been treated to promotional images of the various Copilot+ PC features. Of the Recall feature, the OOBE says it’s “coming soon.”

Image 1 of 3

(Image credit: Tom’s Hardware)

June 19, 2024 at 12:12 AM

Early testing of Asus Vivobook S15 debuts

(Image credit: Future, Windows Central)

A few outlets, includingTom’s Guide andWindows Central , have posted hands-on and early testing of the Asus Vivobook S15. The latter has a full review, which it gave 4.5 out of 5 stars.

Both sites praised the Snapdragon X Elite’s performance (specifically, the 45W TDP X1E78100), as well as the 15.6-inch OLED display. While_Windows Central_ deemed the battery life just “good,”Tom’s Guide called it “mind-blowing.”

TG listed the following benchmark scores, which it says aren’t from its official lab testing:

Geekbench 6 single-core: 2,418
Geekbench 6 multi-core: 14,352
Handbrake: 6:50

All of these scores and times were faster than the M3 MacBook Air the site tested.WC also found it trading blows with certain modern Intel and AMD processors in some tests. Neither site was able to test the NPU, partially due to driver issues.

WC also points out that Arm emulation is better than ever with far more compatibility than before, though it still had issues with Adobe Premiere. Additionally, while a few games loaded well, others couldn’t be installed or would refuse an ARM CPU.

Check out both sites’ reviews and hands-ons for more.

June 19, 2024 at 12:58 AM

Paint’s Cocreator Requires Internet Connection, Doesn’t Follow Instructions

(Image credit: Tom’s Hardware)

We just tried Cocreator, a new Windows Paint feature that is one of the four big new features Microsoft offers exclusively for Copilot+ PCs. It’s a a button in Paint that lets you generate images based on a combination of your drawings and a text prompt.

Color us unimpressed with the feature. Not only does it not follow directions to a T, it also turns out to require an Internet connection as you can see above. While Microsoft never explicitly said that its new Copilot+ features would work offline, it seems ridiculous that these features require a system with a powerful NPU but then send their data to the cloud anyway.

(Image credit: Tom’s Hardware)

So how does it work? We hit the Cocreator button in the upper right corner of the screen, which is not to be confused with the “Image Creator” button that does something similar but is not exclusive to Copilot+ PCs. You’re then shown a sidebar that tells you to “describe the image you’d like to create” and invites you to draw something. So you HAVE TO do both. You can’t just give a text prompt and you can’t just draw something.

Just want to use text prompts? That’s what Image Creator is for, but Image Creator not only requires the cloud, but also has a limited number of tokens you can use (50 by default) before you run out and presumably have to pay for extras.

I wanted to get a picture of a cat driving a car so I drove a circle, a line and an oval with two wheels on it. I used the prompt “cat driving a car,” but Cocreator didn’t exactly follow my directions, showing just a plain cat with something under its arm. When I changed the prompt to “cat on a skateboard,” I got a blurry skateboard below my cat.

(Image credit: Tom’s Hardware)

I had the Windows Task Manager open and watched as the NPU surged to 100 percent for a couple of seconds while Cocreator was redrawing. So it is using the processor, even though it won’t work without an Internet connection.

Incidentally, Cocreator is not close to immediate. When changing the prompt or the drawing, there was a lag of two to three seconds for redrawing. This is undoubtedly better than we’ve seen for something like Stable Diffusion on other systems, but it’s still annoying if you’re in a creative process.

-- Avram Piltch

June 19, 2024 at 2:08 AM

Paint Cocreator will draw a concentration camp, other controversial images

So we asked someone at Qualcomm why Paint Cocreator has to hit the Internet in order to generate an image. After all, isn’t the point of the Snapdragon Elite X and its 45 TOPS NPU that it can do all the image generation offline? The answer we got is that Microsoft wants to make sure that you don’t use prompts that would generate hateful or controversial images.

I don’t understand why Microsoft can’t block bad prompts locally. Lots of people run Stable Diffusion locally and there are some chatbots that run locally so why can’t Windows have the list of forbidden prompts stored locally as well?

But guess what! Even though Cocreator blocks some bad prompts – for example, it won’t draw Hitler – there are plenty of prompts it lets through. Granted, the quality of images is so awful that they may not be recognizable as what you asked for, but it will still do what you asked.

For example, I asked Cocreator to draw a “concentration camp” and it drew a black and white camp area with some people walking through it. Could you tell that this was a concentration camp if you didn’t read the prompt? Probably not.

I also asked for “german camp ww2 jewish” and got a similar image. “German crematorium camp” showed some people lined up outside a smokey building, but if you didn’t know what the prompt was, could you tell?

Image 1 of 3

(Image credit: Tom’s Hardware)

I also asked for Mickey Mouse smoking and, most of the time, I got weird things that were not recognizable as mickey mouse or as smoking. But this image does show Mickey with something long in his hand.

(Image credit: Tom’s Hardware)

Perhaps the worst image I saw was when I entered the prompt “jew angry” or “jewish angry.” Here I got someone who looked literally like the devil. And “jewish boss angry” was a man with a big nose and devil ears.

Image 1 of 2

(Image credit: Tom’s Hardware)

Anyway, if Microsoft’s goal is to prevent its tool from generating biased or embarrassing content, it’s not working.

-- Avram Piltch

June 19, 2024 at 3:40 AM

Snapdragon Laptop bests Intel in Handbrake, Falls behind in Geekbench ML

One real-world test we perform on every laptop is using Handbrake, a popular video encoding app, to transcode a 4K video into a 1080p clip. Fortunately, unlike so many apps, Handbrake already has a native Windows on Arm version so the Snapdragon X Elite chip, specifically an X Elite X1E-80-100, in our Surface Laptop 15-inch doesn’t have to go into emulation to run it.

As elsewhere, we’re comparing the Surface Laptop’s score to that of a ThinkPad X1 Carbon with Core Ultra 7 155H CPU. The Ultra 7 155H is part of Intel’s current-gen Meteor Lake platform, which has a slower NPU than the Snapdragon X Elite (10 TOPS vs 45 TOPS), but has a much faster boost clock (4.8 GHz versus 4.0 GHz).

So how did the Snapdragon X Elite chip fare? It actually was way, way faster than Intel’s offering, completing the task in an average 4 minutes and 55 seconds of~~2 minutes and 39 seconds~~ versus 8 minutes and 57 seconds for the Intel chip. Update: We made a mistake using the wrong source file for the initial round of Snapdragon tests, but that has been corrected. It is still much faster than Intel’s offering, but not quite as fast as we thought.

(Image credit: Tom’s Hardware)

While we wouldn’t call video transcoding an AI workload, it’s clear that Qualcomm does a much better job of handling this task.

Speaking of AI workloads, we also tried Geekbench ML, a relatively new synthetic benchmark that measures inference speed. Geekbench ML claims that it hasone single version for Windows x86 and Windows on Arm. I’m skeptical about whether Geekbench ML is truly optimized for Arm, because normally you need a different installer for Arm compiled apps. If it’s not optimized for Arm, then the Snapdragon chip could be running in emulation mode, which definitely would slow it down.

When we ran Geekbench ML, which measures overall machine learning, we selected DirectML as our inference backend so the system can then at least utilize the GPU rather than the CPU for processing (CPUs are always the slowest). Anyway, in this case, Intel’s chip won 2834 to 2160.

(Image credit: Tom’s Hardware)

Overall, those are promising video numbers for Qualcomm, however.

June 19, 2024 at 5:45 AM

Painful Reminder: Windows 11 is Full of Ads

This probably goes without saying, but I’ll say it here: Windows 11 is festooned with annoying ads. Setting up the Surface Laptop 15 fresh out of the box, I was reminded just how hard Microsoft tries to sell you things.

Today, I was pestered by the operating system three different times to sign up to XBox Game Pass, a $16.99 a month service that lets you play games. First, during the OOBE (out of the box experience), Windows asked me to sign up for the service.

Then, after I first got the system set up, I did not see the Codesigner feature in Windows Paint so I launched the Microsoft Store app in hopes of updating Paint. There was no update there (a reboot later mysteriously made my Paint update), but there was an extremely-annoying, uncloseable ad for – you guessed it – XBox Game pass.

(Image credit: Future)

The Library section of the Microsoft Store is where you can see a list of all your Store apps and which ones need to be updated. Here, covering over part of my list of apps was a Game Pass ad with no close button. I could hit “Claim” and sign up for a free trial (after which I’d have to pay) or “more details” to get more promotional material. I hit “More details” and, even then, a Game Pass promotion remained on screen in my library.

But wait, there’s more! Later, I got a Windows notification asking me to sign up for Game Pass Ultimate. Why can’t Microsoft take no for an answer? I guess it’s hoping to fool users into giving their credit cards for a free 14-day trial and then forgetting to unsubscribe.

(Image credit: Future)

Also, on the lock screen, I got an ad for playing Candy Crush, a preloaded free-to-play game that tries to hook you and get you to spend a lot of money on microtransactions.

(Image credit: Future)

June 19, 2024 at 9:54 AM

Pardon me while I tear my remaining hair out!

One of the big challenges of working with any Windows on Arm device, let alone a brand new one, is compatibility. The thing I want to test most on a Snapdragon X1 Elite-powered laptop is its ability to run real-world AI workloads using models such as Whisper AI and Stable Diffusion.

Stable Diffusion is a popular text-to-image model that can be run locally. Whisper AI is a free speech-to-text model from OpenAI that allows you to feed your computer an audio file and get back a transcript. This is a very helpful use of AI if you’re a journalist like me and you’re doing interviews or watching press announcements. And it’s also great if you’re a student recording a lecture from your teacher.

Whisper AI can run in various programming languages, but the easiest to work with (though not necessarily the fastest) is Python. So, in preparing for our first day of testing, I benchmarked Whisper AI’s base model on our Intel Core Ultra 7 155H-powered ThinkPad. It took 1 minute and 13 seconds to transcribe an 11-minute audio clip. This was using a short Python script that I wrote.

Unfortunately, it’s not so easy to get Whisper AI to run on a Snapdragon laptop, even though Qualcomm has theWhisper Base model in its AI Hub . I just spent the past 4 hours trying to get this model to work, but it looks like – after extensive effort – that it may be impossible.

Theinstructions only describe how to run a quick demo, which transcribes a one-sentence audio clip that Qualcomm has chosen. There doesn’t seem to be a way to deploy and use Qualcomm’s Whisper for an actual Python script in which the user can supply their own audio file. It looks like this was made as a proof of concept and not for serious development as the documentation and examples only show you how to convert the sample audio.

I’m also trying to use the standard Whisper AI python library, which is not maintained by Qualcomm. But that isn’t working on this hardware. It gives a Numpy error because apparently Whisper requries Numpy, a popular Python library, that’s less than version 2. But Numpy less than version 2 won’t install on Arm64 hardware, at least as far as I can tell.

I’m still working on these problems, but the level of difficulty and awful documentation suggests that it may not be possible to build working AI apps in Python for Snapdragon. Qualcomm’s AI hub is full ofproofs of concepts of hundreds of models, from those that to image generation to pose estimation to transcription and yet, instead of showing users how to actually use these models in applications, it offers demos with its own pre-chosen audio, images, etc. Intel actually has useful libraries and documentation, however poorly written, for its OpenVINO libraries. Qualcomm, not so much. I’m very frustrated.

June 19, 2024 at 10:33 PM

On to Day 2 of testing with many problems to solve

(Image credit: Future)

And we’re back. After a frustrating several hours of attempting – and failing – to get Whisper AI running on a Snapdragon-powered laptop, I took a break for the evening and was so annoyed that I had to step away from all screens for about 12 hours.

Today, we’re going to continue our attempts to test the Snapdragon X Elite on a Surface Laptop 15 and a Surface Pro 2-in-1. Overnight, I also learned about a few other frustrations we have to deal with in this process:

Battery tests don’t work: Our lab folks have been attempting to run battery tests on a variety of Snapdragon laptops, including the Surface Pro, and of three different tests, all failed to run. First, our own Battery Informant test, which is written in C# .NET and surfs the web using Edge, failed to open at all. Then they tried PCMark10’s battery test and it refused to run. Finally, theProcyon battery test failed after an hour because it requires you to install Outlook, which we don’t have for Arm (perhaps someone can get it, but it’s not even part of our Office 365 package).
Geekbench ML Not optimized: Qualcomm emailed us to note that Geekbench ML, which we ran yesterday, is not optimized to use the Snapdragon X Elite’s NPU during its testing. So, in looking at the results we posted yesterday – where we speculated that the test wasn’t optimized – I guess we should keep that in mind. Still, it’s a valid test that runs, even if it doesn’t take full advantage of all the processor has to offer.

So what’s on today’s agenda? I’ll be comparing the results from Cinebench 2024, which ran overnight on our Intel Meteor Lake system and also on our Surface Laptop. I’ll be attempting to fix and recompile our battery test, which if it works, will probably take 12+ hours to run. And I’ll be tearing our my hair attempting to get AI workloads to work.

Also, I’ll write about how some of the other Copilot+ exclusive features are running, Studio Effects, for example.

-- Avram Piltch

June 20, 2024 at 1:15 AM

About those Handbrake numbers

So, it looks like the Handbrake numbers we published yesterday were a little off. We use Handbrake 1.8.0, the Arm version for Snapdragon laptops and the x86, 64-bit version for Intel or AMD laptops. We then take a 4K MOV file of Tears of Steel, an open-source movie, and transcode it to 1080p, using Handbrake’s default settings (Fast 1080p, 30).

During yesterday’s run, I accidentally used the pre-compressed version of Tears of Steel on Snapdragon so it was turning a 1080p video into a 1080p video instead of a 4K video into a 4K video. This was a bad mistake, but it doesn’t help that Windows, by default, hides the file extensions (always change this default right away). I otherwise would have noticed that the source file was an MP4 not the 4K MOV file we work with.

Anyhow, I reran the test multiple times on The Surface Laptop 15 and we also got numbers from Surface Pro, which also has a Snapdragon Elite X inside. The numbers are now in the 5-minute range, which is versus 8 to 9 minutes on a laptop with an Intel Core Ultra 7 155H CPU inside.

(Image credit: Future)

Our lab has also recently tested an LG Gram Pro 16 with a Core Ultra 7 155H and a Lenovo Yoga 7 with an AMD Ryzen 8840HS. As you can see in the chart, the AMD chip was quite a bit faster than Intel’s offerings, but still lagged behind Snapdragon by a significant margin.

June 20, 2024 at 2:57 AM

Windows Studio Effects are pretty unimpressive

(Image credit: Future)

With Windows’ controversialRecall feature now postponed , there are only three Copilot+ exclusive features in Windows 11. These are the Codesigner, the image generator that lives in Windows Paint, Live captions with translation and Windows Studio Effects. We talked about Codesigner and its limited utility in some prior posts.

I spent some time fooling around with Studio Effects and I’m not impressed. Studio Effects is a set of webcam special effects that use your local processor instead of the cloud. But these special effects are nothing special and, interestingly, a few of them will run on any Windows 11 PC.

The effects are available to any webcam in any application that accesses your camera. You can get to the settings for Studio Effects either from the Quick Settings menu in the system tray, the Camera section of the overall Settings menu or, if you’re using the Windows Camera app, from a menu within the Windows Camera app (but only in Video mode).

Image 1 of 3

(Image credit: Future)

Studio Effects has five types of settings: Automatic framing, Portrait light, Eye contact, Background effects and Creative filters. Automatic framing , Eye contact and Background effects all appear in the Studio Effects menu on non-Copilot+ PCs as I saw them on our ThinkPad X1 Carbon with Meteor Lake CPU.

(Image credit: Future)

However, the Eye Contact mode has two options on Copilot+ PCs: Standard and Teleprompter mode while it has only one on non-Copilot+ PCs. Either way, in my tests, Eye Contact doesn’t work. What it’s supposed to do is make it look like you are staring at the camera when you are actually looking away.

Imagine if you were looking at a second monitor or at a teleprompter, but you want your viewers to think you are staring at them directly. Nice idea, but when I looked up on the Surface Pro, it didn’t change anything.

(Image credit: Future)

Background effects has either Standard or Portrait blur on both Copilot+ and non-Copilot+ laptops and it seems to work pretty well. I prefer Portrait blur because it makes my torso stand out against the blurry background. However, I’m not too excited about background blur as a feature, because it’s built into so many chat apps already and there are third-party alternatives such as XSplit VCam that will also do this for you.

Automatic framing, another feature you can get elsewhere, works pretty well on either Copilot+ or non-Copilot+ PCs. It simply tries to put you in the center of the frame and it mostly accomplishes that.

Portrait light seemed to have minimal impact on the lighting and colors. In my testing, I couldn’t even see a difference with it on or off.

That leaves us with the three Creative filters, which I’m not sure many people will want to use. There’s Illustrated, Water color and Animated and these basically just make you look a little bit like a painting or a cartoon. the changes can be quite subtle and I noticed them most when looking at the skin on my bald head.

Image 1 of 4

(Image credit: Future)

I think the Animated filter made me look most like a rotoscoped character from_A Scanner Darkly._ That’s interesting, but not exactly a great selling point for a PC.

June 20, 2024 at 5:19 AM

Snapdragon X Elite Aces Cinebench 2024 Test

(Image credit: Future)

Cinebench is one of the most popular synthetic benchmarks around. It uses your CPU cores (or GPU if you choose) to render an image, block by block. We like to run Cinebench twenty times in a row on laptops to see not only how a system performs, but how its performance changes over time as it heats up and cools down.

Unfortunately, at the present moment, Snapdragon X Elite chips don’t properly broadcast their temperature or power consumption to Windows. Therefore, measurement tools such as HWinfo can’t help us log those key metrics while we run Cinebench. We can only look at the scores.

So we downloaded and ran Cinebench 2024, which is available in both x64 versions for Intel / AMD systems and a Windows on Arm version for Snapdragon laptops. Amazingly, in a 20-cycle run of the multicore CPU test, the Snapdragon X Elite-powered Surface Laptop 15 beat the Core Ultra 7 155H-powered ThinkPad X1 Carbon by 72 percent!

The ThinkPad averaged a score of 537, with a low score of 466 on run 1 and a high of 585.9 on run 16. Scores increased dramatically between runs 6 and 9, with the score never dropping below 548 after that. The Surface Laptop had its best score in its second run, hitting 947 and its lowest score on its first run, which was 909. Most runs were in the 910 to 935 range so Snapdragon-powered laptop was not only faster, but more consistent.

(Image credit: Future)

It’s important to note that scores on Cinebench 2024 scores are way lower than Cinebench R23. The older test often returns numbers in the thousands or even in the 15,000 range. However, according to Cinebench 2024’s results database, an Apple M1 Ultra CPU scored 1,625 and a Core i7-1289P-powered laptop returned 433. With that scale in mind, it kind of makes sense that a Core Ultra 7 laptop and a Snapdragon laptop would be between those two other systems.

More about laptops

Asus ROG Zephyrus Duo 15 GX550 Best Gaming Laptops 2024: Tested, benchmarked and reviewed Best Gaming Laptops 2024: Tested, benchmarked and reviewed ](https://tools.techidaily.com ) Real Deals Dell’s 16-inch G16 gaming laptop drops to $949 Dell’s 16-inch G16 gaming laptop drops to $949 ](https://www.tomshardware.com/laptops/gaming-laptops/dells-16-inch-g16-gaming-laptop-drops-to-dollar949 )

Latest

XFX Speedster SWFT210 Radeon RX 7800 XT Core Edition XFX brings back its SWFT 210 cooler for Radeon 7800 XT GPU — XFX Speedster SWFT210 Radeon RX 7800 XT Core Edition rocks dual-fan, minimalist cooler XFX brings back its SWFT 210 cooler for Radeon 7800 XT GPU — XFX Speedster SWFT210 Radeon RX 7800 XT Core Edition rocks dual-fan, minimalist cooler ](https://www.tomshardware.com/pc-components/gpus/xfx-brings-back-its-swft-210-cooler-for-radeon-7800-xt-gpu )

See more latest ►

11 Comments Comment from the forums

Its appears that the gaming performance is not very good right now. Hope it get better with time. There are more reviews incoming as well.

https://www.notebookcheck.com/Asus-Vivobook-S-15-OLED-im-Test-Mit-dem-Snapdragon-X-Elite-in-ein-neues-Notebook-Zeitalter.847338.0.html

Gaming performance is certainly the biggest weak point at the moment and this is where we had the most problems. Basically, it is not easy for users to find out whether there really is a native version or whether the game is emulated.

We were not able to carry out all of our standard tests because there were some problems. On the one hand, there were image errors at higher details (e.g. Total War Pharaoh) or crashes (e.g. X-Plane, Shadow of the Tomb Raider), but many titles simply did not start at all. These include, for example, F1 23, F1 24, Prince of Persia: The Los Crown or Far Cry 5. In Cyberpunk 2077, the game also occasionally crashed. We will be testing more titles in the next few days.

The Qualcomm chip supports what is known as automatic super resolution. Basically, the game is upscaled with AI support, so you can select a lower resolution, theoretically without any graphical loss. We will also be testing this in the next few days.

The gaming performance falls behind the other iGPUs and in general it must be said that the Snapdragon laptops are only suitable for gaming to a very limited extent.

https://www.techpowerup.com/img/ZdVP4JWfzeQmN9EI.jpg
Reply

I guess more reviews are gonna come at a later date.

1803050279363023302 View: https://x.com/zacbowden/status/1803050279363023302

But here are some of them:

uyHs5-XYh6s:32 View: https://www.youtube.com/watch?v=uyHs5-XYh6s&t=32s&ab\_channel=MatthewMoniz
rSx0WZfDbE0:59 View: https://www.youtube.com/watch?v=rSx0WZfDbE0&t=59s&ab\_channel=Dave2D
svZHybT9Jg4 View: https://www.youtube.com/watch?v=svZHybT9Jg4&ab\_channel=TechTablets
https://i.imgur.com/foboDSs.png
Reply

35below0
Shat’s with jews and concentration camps in the cocreator article?? Couldn’t think of something else controversial that isn’t playing with genocide?
Like images of a certain Prophet for example? Or you know, that old classic hard core photography of the human form.

Actually, i better not give any ideas

Actually, actually, now i’m wondering whether it could draw a censored protest in China, since all possible sources of information are supressed? Or back to the prophet again, because how would one know what he looks like?
Or something useful, like a pallete swap that is colorblind friendly. Maybe that one would be easy, maybe not.
Reply

I guess the battery life seems to be the strongest factor for these Snapdragon X CPU Laptops. But it’s frustrating to know that “Cocreator” won’t work without an Internet connection.

“So we’re in the middle of setting up our Surface Laptop 15-inch with Snapdragon Elite X CPU and it is taking . . . forever.”

Any luck or progress yet ?
Reply

Notton
Where are the naysayers that were out in droves in this thread?
https://www.tomshardware.com/laptops/snapdragon-x-elite-in-the-wild-is-allegedly-slower-than-iphone-12-first-benchmarks-of-samsung-book4-edge-disappoint
Reply
Alvar “Miles” Udell

Painful Reminder: Windows 11 is Full of Ads

Friendly reminder: They’re easily disabled.
Reply

vijosef
This link is broken on the page, and there is no link to here
BETa8Cy
Reply
KnightShadey

Metal Messiah. said:

Its appears that the gaming performance is not very good right now. Hope it get better with time. There are more reviews incoming as well.

https://www.notebookcheck.com/Asus-Vivobook-S-15-OLED-im-Test-Mit-dem-Snapdragon-X-Elite-in-ein-neues-Notebook-Zeitalter.847338.0.html

The disappointing thing is that NB didn’t test it against more U series laptops. All the intel’s are H series, which is fine if your only metric is price not efficient.
Price will change over time, but even a i7 150U would’ve been better than a 155H for comparison for this type of laptop.

Looks like they have no 150U nor 120U in their benchmark history, but if you manually compare results to the nearby Ci7 1365U, it looks to be very similar in performance, beating it in tests like Blender 727s vs 545s. Unfortunately many of the other tests are not compatible for comparison.

https://www.notebookcheck.net/Intel-Core-i7-1365U-Processor-Benchmarks-and-Specs.678465.0.html
In comparison to its segment, looks ‘OK’, not bad, but not as impressive as hyped.
Reply