美国时间10月22日，Anthropic推出重大更新！其中包括升级版的 Claude 3.5 Sonnet 以及一款新的 Claude 3.5 Haiku 模型，除此之外，这次还带来了全新功能Computer Use，这一功能支持AI像人类一样操作计算机，可以遵循用户的命令在计算机屏幕上移动光标，点击相关位置，并通过虚拟键盘输入信息，模拟人们与自己计算机的交互方式。今天我们就教一教大家，如何通过302.AI平台来进行测试。
Computer Use的原理
Computer Use的原理就是：本地程序每次对电脑屏幕进行截屏，通过API传给AI，AI通过多模态识别能力判断鼠标和屏幕状态，返回操作命令，本地程序接受命令，对电脑进行操作，如此反复。每一个操作都要截图一次。
如何使用官方Demo
官方demo的原理是利用docker在本地虚拟了一个linux环境，通过浏览器展示，同时用Claude来控制这个虚拟linux。官方demo的优势是界面比较直观，linux环境简单可控，比较容易展示各种功能。劣势是仅仅只能作为demo，实际应用价值有限。
具体操作步骤如下（以下均在windows环境，如果是Mac，命令可能会有少许不同）：
1、下载docker程序并安装：https://www.docker.com/
2、运行CMD
3、设置环境变量，复制下面的代码，点击回车
（其中API KEY在302后台生成，以sk开头）
set 302_BASE_URL=https://api.302.ai
set 302_API_KEY=sk-xxx
4、下载并运行官方demo，复制以下代码，点击回车
docker run ^
-e ANTHROPIC_API_KEY=%302_API_KEY% ^
-e ANTHROPIC_BASE_URL=%302_BASE_URL% ^
-v %USERPROFILE%/.anthropic:/home/computeruse/.anthropic ^
-p 5900:5900 ^
-p 8501:8501 ^
-p 6080:6080 ^
-p 8080:8080 ^
-it ghcr.io/anthropics/anthropic-quickstarts:computer-use-demo-latest
（第一次会下载一些文件，时间会久一点，偶尔会因为网络问题下载不了，多试几次）
5、运行成功，显示如下：
点击http://localhost:8080 即可体验
如何使用独立程序
github链接：https://github.com/302team/302-agent.exe
（这里是我们修改的版本，将API地址改为了302.AI的地址）
独立程序的原理是使用NodeJS开发了一款桌面端的软件，它能够直接对真实的桌面进行控制。独立程序的优势是可以对真实的桌面环境进行控制，让人有种AI控制电脑的创新体验感，劣势是由于每个人的桌面环境都非常复杂，这导致demo演示中成功率非常低。尤其是这个开源程序只会使用firefox浏览器。
具体操作步骤如下（以下均在Mac环境）：
1、下载并安装NodeJS环境：https://nodejs.org/en/download/prebuilt-installer
2、打开终端，按顺序输入下面的4条命令
git clone https://github.com/302team/302-agent.exe下载软件包
cd 302-agent.exe进入软件包
npm install进行安装
ANTHROPIC_API_KEY="sk-xxxx" npm start3、输入API KEY，在302后台生成，以sk开头。之后每次运行只需要运行第二步和最后一步。
此时应该会看到一个Agent.exe
第一次打开会需要授予比较多的权限，授予后即可使用。此时就可以使用了
Computer Use的成功案例：
最后来看下通过302.AI生成的Demo，（以下demo为了方便观看，作了倍速处理）：
1、搜索信息：当发送指令：“打开浏览器，搜索AI相关新闻”后，可以看到AI打开了一个网页浏览器，并在搜索引擎输入了关键词“AI新闻”，随后打开了一篇AI相关的新闻报道，最后还在对话框对搜索结果做了简单的总结：
2、玩游戏：当发送指令：“打开浏览器，进入五子棋页面，并开始游戏”，AI就会一步步找到提供的五子棋游戏网址，并点击开始游戏后，自己开始玩游戏：
3、内容抓取：可以让它直接进入网页“302.ai”，并总结网页内容反馈到对话框中：
4、AI和AI对话：在上一个demo的基础上，还可以一步步引导，最后实现利用computer use让ai和ai对话：
Computer Use的失败案例：
1、数值计算：输入指令让它用计算器计算10000+1000时，计算器中得出的结果是200，原因大概率是重复点击的时候，丢失了点击。而AI模型识别到了这一错误，在对话框中输出了正确结果：11000，但这个属于作弊了。以此类推，遇到重复点击的情况，AI很有可能会出现错误，而在生产环境中，重复性的输入和点击是非常常见的。
2、信息查找：输入的指令是：打开浏览器，进入网页”302.ai”，点击价格页面，我想要使用图片放大功能，价格是多少?
在前几步，有正确的打开价格表，但是在最后查询价格的时候，出现了幻觉，答案是
而大家都知道，302.AI是一个没有任何订阅的平台，只支持按需付费，这里完全是AI的幻觉，产生了虚假信息，对人进行了误导。
Computer Use现在存在的问题
虽然在Anthropic发布Computer Use功能后，众多博主纷纷夸大其词地赞扬这一功能，但是经过302的实测，发现存在了三个主要的问题：
1、速度慢：做一步想一步，基本都要间隔10-20s，效率极低，不知Claude未来是否可以实现提前规划几步？
2、错误率高：拿计算器作为例子，让它计算10000+1000，可能会少打几个0；而且对于复杂流程，容易错乱和产生幻觉。
3、费用高：每次操作都要截屏回传进行分析，且都是几千上万的token消耗；
总的来说，慢，错，贵是三个大问题，慢和贵可以通过端侧大模型解决，但错误率的话，Claude作为最强的模型，错误率都如此之高，可想而知端侧大模型更是困难，所以在现阶段是不可能三角都兼得的。
在现阶段，Computer Use这一功能其实并没有任何实际性生产力，仅可以作为一个Demo展示。但是随着时间的发展，这个功能一定会越来越好用，从15%的操作成功率（Anthropic官方数据），直到超越人类的操作成功率。
对未来的展望
最后再展望下Computer Use的未来，RPA一直是生产环境的一个刚需，Anthropic今天正式打开了AI-RPA的序幕。未来大概率会出现完全为了操作电脑而训练的端侧模型，云端模型仅仅做一次大的路径规划，其余全部交给端侧模型去处理，端侧模型可以对一些小的错误进行判断和重试，无需云端模型参与，简单理解就是有点类似自动驾驶，云端模型计算导航路线，本地模型进行驾驶。
👉立即注册免费试用302.AI，开启你的AI之旅！👈
为什么选择302.AI？
● 灵活付费：无需月费，按需付费，成本可控
● 丰富功能：从文字、图片到视频，应有尽有，满足多种场景需求
● 开源生态：支持开发者深度定制，打造专属AI应用
● 易用性：界面友好，操作简单，快速上手
Comments(74)
I and my guys came reading through the excellent thoughts found on the website and so unexpectedly I got a horrible suspicion I never expressed respect to the web blog owner for those techniques. Most of the guys were definitely stimulated to see all of them and already have undoubtedly been having fun with them. We appreciate you genuinely quite kind as well as for considering these kinds of impressive information most people are really wanting to be aware of. My very own sincere regret for not saying thanks to you earlier.
Hi, I think your site might be having browser compatibility issues. When I look at your website in Safari, it looks fine but when opening in Internet Explorer, it has some overlapping. I just wanted to give you a quick heads up! Other then that, fantastic blog!
I do enjoy the way you have framed this particular challenge and it really does present me some fodder for consideration. However, through what I have seen, I simply hope when other comments pack on that folks continue to be on point and don’t start on a soap box associated with the news of the day. Still, thank you for this fantastic point and even though I can not necessarily go along with this in totality, I regard your perspective.
Once I initially commented I clicked the -Notify me when new feedback are added- checkbox and now every time a remark is added I get 4 emails with the same comment. Is there any way you’ll be able to remove me from that service? Thanks!
Excellent site. Plenty of helpful information here. I am sending it to some friends ans also sharing in delicious. And obviously, thank you to your sweat!
I enjoy looking through and I think this website got some truly utilitarian stuff on it! .
Hi there! This post couldn’t be written any better! Reading through this post reminds me of my previous room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thank you for sharing!
Unquestionably imagine that which you stated. Your favorite reason seemed to be on the web the easiest factor to be mindful of. I say to you, I certainly get annoyed whilst other folks think about issues that they just do not recognise about. You controlled to hit the nail upon the top as neatly as outlined out the entire thing with no need side-effects , folks can take a signal. Will likely be again to get more. Thank you
Hey just wanted to give you a brief heads up and let you know a few of the pictures aren’t loading properly. I’m not sure why but I think its a linking issue. I’ve tried it in two different internet browsers and both show the same results.
Can I simply say what a aid to seek out someone who really is aware of what theyre talking about on the internet. You definitely know learn how to convey an issue to light and make it important. More individuals need to learn this and understand this aspect of the story. I cant imagine youre not more popular since you definitely have the gift.
I¦ve recently started a blog, the info you offer on this website has helped me greatly. Thank you for all of your time & work.
Very interesting points you have remarked, thankyou for posting.
I definitely wanted to post a quick word to be able to appreciate you for all the marvelous instructions you are posting here. My long internet investigation has at the end been honored with incredibly good details to share with my companions. I would point out that many of us readers actually are quite endowed to live in a perfect community with so many special people with great tips and hints. I feel pretty lucky to have come across the weblog and look forward to really more fun times reading here. Thanks a lot once again for a lot of things.
It is actually a nice and helpful piece of info. I am satisfied that you shared this useful info with us. Please keep us up to date like this. Thanks for sharing.
Hey! Someone in my Myspace group shared this site with us so I came to take a look. I’m definitely loving the information. I’m bookmarking and will be tweeting this to my followers! Great blog and excellent style and design.
Hey very nice website!! Man .. Excellent .. Amazing .. I will bookmark your web site and take the feeds also…I’m happy to find numerous useful info here in the post, we need develop more strategies in this regard, thanks for sharing. . . . . .
I view something truly special in this web site.
Hello very nice web site!! Man .. Excellent .. Amazing .. I’ll bookmark your site and take the feeds additionally…I’m glad to find numerous useful information here in the post, we’d like work out more strategies on this regard, thank you for sharing. . . . . .
fantástico este conteúdo. Gostei muito. Aproveitem e vejam este site. informações, novidades e muito mais. Não deixem de acessar para aprender mais. Obrigado a todos e até a próxima. :)
I visited a lot of website but I think this one contains something extra in it in it
I have been absent for some time, but now I remember why I used to love this website. Thanks , I¦ll try and check back more frequently. How frequently you update your website?
Hello there! Would you mind if I share your blog with my facebook group? There’s a lot of people that I think would really appreciate your content. Please let me know. Thank you
I was studying some of your posts on this site and I think this site is real instructive! Retain posting.
I’d must test with you here. Which is not one thing I often do! I enjoy reading a publish that will make people think. Also, thanks for permitting me to comment!
You actually make it seem so easy with your presentation but I find this matter to be actually something which I think I would never understand. It seems too complicated and extremely broad for me. I am looking forward for your next post, I will try to get the hang of it!
I have been exploring for a little bit for any high quality articles or blog posts in this sort of space . Exploring in Yahoo I eventually stumbled upon this site. Reading this information So i am glad to show that I’ve a very just right uncanny feeling I discovered just what I needed. I such a lot definitely will make certain to do not forget this web site and give it a glance on a relentless basis.
You made several fine points there. I did a search on the subject and found a good number of persons will go along with with your blog.
The Pink Salt Trick is a minimalist but effective morning routine: Just drink a glass of lukewarm water mixed with a pinch of Himalayan pink salt as soo
Everything is very open and very clear explanation of issues. was truly information. Your website is very useful. Thanks for sharing.
The Pink Salt Trick is a minimalist but effective morning routine: Just drink a glass of lukewarm water mixed with a pinch of Himalayan pink salt as soo
Excellent post. I was checking constantly this blog and I am impressed! Very useful information particularly the last part :) I care for such information much. I was seeking this certain info for a long time. Thank you and good luck.
Hey there, You’ve done an incredible job. I will certainly digg it and in my view recommend to my friends. I’m sure they’ll be benefited from this site.
Hey there just wanted to give you a brief heads up and let you know a few of the pictures aren’t loading properly. I’m not sure why but I think its a linking issue. I’ve tried it in two different web browsers and both show the same results.
hi!,I like your writing so much! share we communicate more about your article on AOL? I require a specialist on this area to solve my problem. Maybe that’s you! Looking forward to see you.
Hey there just wanted to give you a quick heads up and let you know a few of the images aren’t loading properly. I’m not sure why but I think its a linking issue. I’ve tried it in two different web browsers and both show the same outcome.
Wow, marvelous blog layout! How long have you been blogging for? you made blogging look easy. The overall look of your site is wonderful, as well as the content!
I was reading through some of your posts on this website and I think this web site is very informative ! Keep putting up.
Hello, Neat post. There’s an issue together with your site in web explorer, would test thisK IE nonetheless is the market leader and a huge portion of folks will pass over your wonderful writing because of this problem.
Please let me know if you’re looking for a author for your site. You have some really good posts and I think I would be a good asset. If you ever want to take some of the load off, I’d really like to write some material for your blog in exchange for a link back to mine. Please send me an email if interested. Kudos!
Excellent goods from you, man. I have understand your stuff previous to and you’re just too magnificent. I actually like what you have acquired here, really like what you are stating and the way in which you say it. You make it entertaining and you still take care of to keep it wise. I can’t wait to read far more from you. This is actually a great site.
I genuinely enjoy examining on this web site, it holds good content. “Words are, of course, the most powerful drug used by mankind.” by Rudyard Kipling.
I like this blog very much, Its a real nice billet to read and receive information.
Hello. remarkable job. I did not imagine this. This is a excellent story. Thanks!
Only wanna input that you have a very decent site, I like the design it actually stands out.
I got what you mean , thankyou for putting up.Woh I am delighted to find this website through google.
Magnificent website. Lots of helpful info here. I am sending it to several buddies ans also sharing in delicious. And certainly, thank you for your sweat!
It is in point of fact a nice and useful piece of info. I’m happy that you shared this helpful information with us. Please keep us up to date like this. Thank you for sharing.
naturally like your website but you have to take a look at the spelling on several of your posts. Many of them are rife with spelling issues and I find it very bothersome to inform the truth then again I?¦ll surely come back again.
Thank you for sharing superb informations. Your web site is so cool. I’m impressed by the details that you’ve on this blog. It reveals how nicely you understand this subject. Bookmarked this web page, will come back for more articles. You, my friend, ROCK! I found simply the info I already searched all over the place and just could not come across. What a perfect web site.
I keep listening to the news bulletin speak about receiving boundless online grant applications so I have been looking around for the finest site to get one. Could you tell me please, where could i get some?
Some really interesting information, well written and broadly user genial.
Sweet website , super design and style, rattling clean and apply friendly.
I am glad to be a visitant of this sodding web blog! , thanks for this rare info ! .
Hello. impressive job. I did not anticipate this. This is a impressive story. Thanks!
Hello! I could have sworn I’ve been to this blog before but after browsing through some of the post I realized it’s new to me. Anyways, I’m definitely happy I found it and I’ll be book-marking and checking back frequently!
Hello! I just would like to give a huge thumbs up for the great info you have here on this post. I will be coming back to your blog for more soon.
I must show my gratitude for your generosity in support of men and women that should have help on that niche. Your real dedication to getting the solution up and down ended up being especially interesting and have surely allowed women just like me to realize their desired goals. The helpful key points indicates a great deal to me and a whole lot more to my fellow workers. Warm regards; from each one of us.
This is very interesting, You are a very skilled blogger. I’ve joined your rss feed and look forward to seeking more of your fantastic post. Also, I’ve shared your site in my social networks!
It?¦s really a nice and useful piece of info. I?¦m happy that you just shared this helpful info with us. Please keep us informed like this. Thank you for sharing.
Hmm is anyone else encountering problems with the pictures on this blog loading? I’m trying to find out if its a problem on my end or if it’s the blog. Any feed-back would be greatly appreciated.
This blog is definitely rather handy since I’m at the moment creating an internet floral website – although I am only starting out therefore it’s really fairly small, nothing like this site. Can link to a few of the posts here as they are quite. Thanks much. Zoey Olsen
Hey There. I found your blog using msn. That is a really neatly written article. I’ll be sure to bookmark it and return to read more of your helpful info. Thank you for the post. I will definitely comeback.
I will immediately snatch your rss as I can’t in finding your e-mail subscription hyperlink or newsletter service. Do you have any? Kindly let me realize in order that I may subscribe. Thanks.
Yay google is my world beater helped me to find this great site! .
Glad to be one of many visitants on this awing internet site : D.
As a Newbie, I am constantly searching online for articles that can benefit me. Thank you
Hi there would you mind letting me know which web host you’re working with? I’ve loaded your blog in 3 completely different internet browsers and I must say this blog loads a lot quicker then most. Can you recommend a good hosting provider at a honest price? Thank you, I appreciate it!
You really make it appear so easy together with your presentation but I to find this topic to be really something which I believe I might never understand. It sort of feels too complicated and extremely extensive for me. I’m looking forward in your subsequent post, I?¦ll attempt to get the hold of it!
I believe this site has got some very excellent information for everyone. “Few friendships would survive if each one knew what his friend says of him behind his back.” by Blaise Pascal.
As a Newbie, I am always exploring online for articles that can be of assistance to me. Thank you
Good V I should certainly pronounce, impressed with your site. I had no trouble navigating through all tabs and related info ended up being truly simple to do to access. I recently found what I hoped for before you know it at all. Quite unusual. Is likely to appreciate it for those who add forums or anything, web site theme . a tones way for your client to communicate. Nice task..
There are definitely lots of particulars like that to take into consideration. That is a great point to bring up. I provide the thoughts above as common inspiration but clearly there are questions just like the one you deliver up the place an important thing will be working in sincere good faith. I don?t know if finest practices have emerged around things like that, but I’m sure that your job is clearly recognized as a good game. Each boys and girls really feel the impact of just a moment’s pleasure, for the remainder of their lives.
I really like your writing style, wonderful information, appreciate it for putting up : D.
I am not sure where you are getting your info, but good topic. I needs to spend some time learning much more or understanding more. Thanks for wonderful info I was looking for this information for my mission.