使用AI进行剧作家测试。
auto-playwright依赖性: npm install auto-playwright -D.env文件中: export OPENAI_API_KEY= ' sk-..."auto功能: import { test , expect } from "@playwright/test" ;
import { auto } from "auto-playwright" ;
test ( "auto Playwright example" , async ( { page } ) => {
await page . goto ( "/" ) ;
// `auto` can query data
// In this case, the result is plain-text contents of the header
const headerText = await auto ( "get the header text" , { page , test } ) ;
// `auto` can perform actions
// In this case, auto will find and fill in the search text input
await auto ( `Type " ${ headerText } " in the search box` , { page , test } ) ;
// `auto` can assert the state of the website
// In this case, the result is a boolean outcome
const searchInputHasHeaderText = await auto ( `Is the contents of the search box equal to " ${ headerText } "?` , { page , test } ) ;
expect ( searchInputHasHeaderText ) . toBe ( true ) ;
} ) ;包括带有连接到Azure OpenAI所需的值的Stepoptions类型。
import { test , expect } from "@playwright/test" ;
import { auto } from "auto-playwright" ;
import { StepOptions } from "../src/types" ;
const apiKey = "apikey" ;
const resource = "azure-resource-name" ;
const model = "model-deployment-name" ;
const options : StepOptions = {
model : model ,
openaiApiKey : apiKey ,
openaiBaseUrl : `https:// ${ resource } .openai.azure.com/openai/deployments/ ${ model } ` ,
openaiDefaultQuery : { 'api-version' : "2023-07-01-preview" } ,
openaiDefaultHeaders : { 'api-key' : apiKey }
} ;
test ( "auto Playwright example" , async ( { page } ) => {
await page . goto ( "/" ) ;
// `auto` can query data
// In this case, the result is plain-text contents of the header
const headerText = await auto ( "get the header text" , { page , test } , options ) ;
// `auto` can perform actions
// In this case, auto will find and fill in the search text input
await auto ( `Type " ${ headerText } " in the search box` , { page , test } , options ) ;
// `auto` can assert the state of the website
// In this case, the result is a boolean outcome
const searchInputHasHeaderText = await auto ( `Is the contents of the search box equal to " ${ headerText } "?` , { page , test } , options ) ;
expect ( searchInputHasHeaderText ) . toBe ( true ) ;
} ) ; 至少, auto函数需要一个纯文本提示,并且一个包含您的page和test (可选)对象的参数。
auto ( "<your prompt>" , { page , test } ) ;在没有test参数的情况下运行:
import { chromium } from "playwright" ;
import { auto } from "auto-playwright" ;
( async ( ) => {
const browser = await chromium . launch ( { headless : true } ) ;
const context = await browser . newContext ( ) ;
const page = await context . newPage ( ) ;
// Navigate to a website
await page . goto ( "https://www.example.com" ) ;
// `auto` can query data
// In this case, the result is plain-text contents of the header
const res = await auto ( "get the header text" , { page } ) ;
// use res.query to get a query result.
console . log ( res ) ;
await page . close ( ) ;
} ) ( ) ;您可以将debug属性作为第三个参数传递给auto函数。这将打印提示,并由OpenAI执行的命令。
await auto ( "get the header text" , { page , test } , { debug : true } ) ;您还可以设置环境变量AUTO_PLAYWRIGHT_DEBUG=true ,这将启用所有auto调用的调试。
export AUTO_PLAYWRIGHT_DEBUG=true每个剧作家都支持的每个浏览器。
还有其他选项可以作为第三个参数传递:
const options = {
// If true, debugging information is printed in the console.
debug : boolean ,
// The OpenAI model (https://platform.openai.com/docs/models/overview)
model : "gpt-4-1106-preview" ,
// The OpenAI API key
openaiApiKey : 'sk-...' ,
} ;
auto ( "<your prompt>" , { page , test } , options ) ; 根据动作type (由auto函数推断),有不同的行为和返回类型。
操作(例如“单击”)是与该页面的一些模拟用户交互,例如单击链接。动作将返回`'''
try {
await auto ( "click the link" , { page , test } ) ;
} catch ( e ) {
console . error ( "failed to click the link" ) ;
}查询将以字符串的形式从页面返回请求的数据,例如
const linkText = await auto ( "Get the text of the first link" , { page , test } ) ;
console . log ( "The link text is" , linkText ) ;断言是一个将返回true还是false问题,例如
const thereAreThreeLinks = await auto ( "Are there 3 links on the page?" , {
page ,
test ,
} ) ;
console . log ( `"There are 3 links" is a ${ thereAreThreeLinks } statement` ) ; | 方面 | 常规方法 | 与自动剧作家一起测试 |
|---|---|---|
| 与标记结合 | 与应用程序的标记密切相关。 | 消除选择器的使用;动作由AI助手在运行时确定。 |
| 实施速度 | 由于需要为每个操作进行精确代码翻译,因此实现速度较慢。 | 使用简单的纯文本说明进行动作和断言的快速测试。 |
| 处理复杂的方案 | 自动化复杂方案是具有挑战性的,容易出现频繁的失败。 | 通过关注预期的测试结果来促进复杂方案的测试。 |
| 测试写作时间 | 仅在功能的完整开发后才能编写测试。 | 启用测试驱动的开发(TDD)方法,允许与功能开发或之前或之前的测试写作。 |
locator.blurlocator.boundingBoxlocator.checklocator.clearlocator.clicklocator.countlocator.filllocator.getAttributelocator.innerHTMLlocator.innerTextlocator.inputValuelocator.isCheckedlocator.isEditablelocator.isEnabledlocator.isVisiblelocator.textContentlocator.uncheckpage.goto添加新操作很容易:只需更新src/completeTask.ts中的functions即可。
这个库是免费的。但是,使用OpenAi有一些成本。您可以在此处找到有关定价的更多信息:https://openai.com/pricing/。
以https://ray.run/为例,使用GPT-4 Turbo(使用GPT-3.5 Turbo)运行测试步骤的成本约为0.01美元。
低成本部分是因为auto-playwright使用HTML消毒来减少有效载荷大小,例如,下面的有效载荷将提交https://ray.run/。
自然,价格会根据有效载荷而变化很大。
< div class =" cYdhWw dKnOgO geGbZz bGoBgk jkEels " >
< div class =" kSmiQp fPSBzf bnYmbW dXscgu xJzwH jTWvec gzBMzy " >
< h1 class =" fwYeZS fwlORb pdjVK bccLBY fsAQjR fyszFl WNJim fzozfU " >
Learn Playwright
</ h1 >
< h2 class =" cakMWc ptfck bBmAxp hSiiri xJzwS gnfYng jTWvec fzozfU " >
Resources for learning end-to-end testing using Playwright automation
framework
</ h2 >
< div
class =" bLTbYS gvHvKe cHEBuD ddgODW jsxhGC kdTEUJ ilCTXp iQHbtH yuxBn ilIXfy gPeiPq ivcdqp isDTsq jyZWmS ivdkBK cERSkX hdAwi ezvbLT jNrAaV jsxhGJ fzozCb "
> </ div >
</ div >
< div class =" cYdhWw dpjphg cqUdSC fasMpP " >
< a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /blog "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Blog </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Learn in depth subjects about end-to-end testing. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /ask "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Ask AI </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Ask ChatGPT Playwright questions. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /tools "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Dev Tools </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > All-in-one toolbox for QA engineers. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /jobs "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > QA Jobs </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Handpicked QA and Automation opportunities. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /questions "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Questions </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Ask AI answered questions about Playwright. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /discord-forum "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Discord Forum </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Archive of Discord Forum posts about Playwright. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /videos "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Videos </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Tutorials, conference talks, and release videos. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /browser-extension "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Browser Extension </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > GUI for generating Playwright locators. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /wiki "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > QA Wiki </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Definitions of common end-to-end testing terms. </ p >
</ div > </ a
>
</ div >
< div
class =" kSmiQp fPSBzf pKTba eTDpsp legDhJ hSiiri hdaZLM jTWvec gzBMzy bGySga fzoybr "
>
< p class =" dXhlDK leOtqz glpWRZ fNCcFz " >
Use < kbd class =" bWhrAL XAzZz cakMWc bUyOMB bmOrOm fyszFl dTmriP " > ⌘ </ kbd > +
< kbd > k </ kbd > + "Tools" to quickly access all tools.
</ p >
</ div >
</ div > auto功能使用Sanitize-HTML在将页面的HTML消毒之前,然后将其发送到OpenAI。这样做是为了降低成本并提高生成的文本的质量。
该项目从Zerostep汲取了灵感。 Zerostep提供了类似的API,但通过其专有后端具有更强大的实现。 Auto Playwright的创建是为了探索Zerostep的基础技术,并为其软件的开源版本建立基础。对于生产环境,我建议选择Zerostep。
这是自动剧作家和Zerostep的并排比较:
| 标准 | 自动剧作家 | Zerostep |
|---|---|---|
| 使用OpenAI API | 是的 | 否1 |
| 使用普通文本提示 | 是的 | 不 |
使用functions SDK | 是的 | 不 |
| 使用HTML消毒 | 是的 | 不 |
| 使用剧作家API | 是的 | 否2 |
| 使用屏幕截图 | 不 | 是的 |
| 使用队列 | 不 | 是的 |
| 使用Websockets | 不 | 是的 |
| 快照 | html | dom |
| 实现并行性 | 不 | 是的 |
| 允许滚动 | 不 | 是的 |
| 提供固定装置 | 不 | 是的 |
| 执照 | 麻省理工学院 | 麻省理工学院 |
MIT License
Copyright (c) 2023 Reflect Software Inc
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
使用Zerostep专有API。 ↩
使用一些剧作家API,但主要依赖于Chrome DevTools协议(CDP)。 ↩