使用AI進行劇作家測試。
auto-playwright依賴性: npm install auto-playwright -D.env文件中: export OPENAI_API_KEY= ' sk-..."auto功能: import { test , expect } from "@playwright/test" ;
import { auto } from "auto-playwright" ;
test ( "auto Playwright example" , async ( { page } ) => {
await page . goto ( "/" ) ;
// `auto` can query data
// In this case, the result is plain-text contents of the header
const headerText = await auto ( "get the header text" , { page , test } ) ;
// `auto` can perform actions
// In this case, auto will find and fill in the search text input
await auto ( `Type " ${ headerText } " in the search box` , { page , test } ) ;
// `auto` can assert the state of the website
// In this case, the result is a boolean outcome
const searchInputHasHeaderText = await auto ( `Is the contents of the search box equal to " ${ headerText } "?` , { page , test } ) ;
expect ( searchInputHasHeaderText ) . toBe ( true ) ;
} ) ;包括帶有連接到Azure OpenAI所需的值的Stepoptions類型。
import { test , expect } from "@playwright/test" ;
import { auto } from "auto-playwright" ;
import { StepOptions } from "../src/types" ;
const apiKey = "apikey" ;
const resource = "azure-resource-name" ;
const model = "model-deployment-name" ;
const options : StepOptions = {
model : model ,
openaiApiKey : apiKey ,
openaiBaseUrl : `https:// ${ resource } .openai.azure.com/openai/deployments/ ${ model } ` ,
openaiDefaultQuery : { 'api-version' : "2023-07-01-preview" } ,
openaiDefaultHeaders : { 'api-key' : apiKey }
} ;
test ( "auto Playwright example" , async ( { page } ) => {
await page . goto ( "/" ) ;
// `auto` can query data
// In this case, the result is plain-text contents of the header
const headerText = await auto ( "get the header text" , { page , test } , options ) ;
// `auto` can perform actions
// In this case, auto will find and fill in the search text input
await auto ( `Type " ${ headerText } " in the search box` , { page , test } , options ) ;
// `auto` can assert the state of the website
// In this case, the result is a boolean outcome
const searchInputHasHeaderText = await auto ( `Is the contents of the search box equal to " ${ headerText } "?` , { page , test } , options ) ;
expect ( searchInputHasHeaderText ) . toBe ( true ) ;
} ) ; 至少, auto函數需要一個純文本提示,並且一個包含您的page和test (可選)對象的參數。
auto ( "<your prompt>" , { page , test } ) ;在沒有test參數的情況下運行:
import { chromium } from "playwright" ;
import { auto } from "auto-playwright" ;
( async ( ) => {
const browser = await chromium . launch ( { headless : true } ) ;
const context = await browser . newContext ( ) ;
const page = await context . newPage ( ) ;
// Navigate to a website
await page . goto ( "https://www.example.com" ) ;
// `auto` can query data
// In this case, the result is plain-text contents of the header
const res = await auto ( "get the header text" , { page } ) ;
// use res.query to get a query result.
console . log ( res ) ;
await page . close ( ) ;
} ) ( ) ;您可以將debug屬性作為第三個參數傳遞給auto函數。這將打印提示,並由OpenAI執行的命令。
await auto ( "get the header text" , { page , test } , { debug : true } ) ;您還可以設置環境變量AUTO_PLAYWRIGHT_DEBUG=true ,這將啟用所有auto調用的調試。
export AUTO_PLAYWRIGHT_DEBUG=true每個劇作家都支持的每個瀏覽器。
還有其他選項可以作為第三個參數傳遞:
const options = {
// If true, debugging information is printed in the console.
debug : boolean ,
// The OpenAI model (https://platform.openai.com/docs/models/overview)
model : "gpt-4-1106-preview" ,
// The OpenAI API key
openaiApiKey : 'sk-...' ,
} ;
auto ( "<your prompt>" , { page , test } , options ) ; 根據動作type (由auto函數推斷),有不同的行為和返回類型。
操作(例如“單擊”)是與該頁面的一些模擬用戶交互,例如單擊鏈接。動作將返回`'''
try {
await auto ( "click the link" , { page , test } ) ;
} catch ( e ) {
console . error ( "failed to click the link" ) ;
}查詢將以字符串的形式從頁面返回請求的數據,例如
const linkText = await auto ( "Get the text of the first link" , { page , test } ) ;
console . log ( "The link text is" , linkText ) ;斷言是一個將返回true還是false問題,例如
const thereAreThreeLinks = await auto ( "Are there 3 links on the page?" , {
page ,
test ,
} ) ;
console . log ( `"There are 3 links" is a ${ thereAreThreeLinks } statement` ) ; | 方面 | 常規方法 | 與自動劇作家一起測試 |
|---|---|---|
| 與標記結合 | 與應用程序的標記密切相關。 | 消除選擇器的使用;動作由AI助手在運行時確定。 |
| 實施速度 | 由於需要為每個操作進行精確代碼翻譯,因此實現速度較慢。 | 使用簡單的純文本說明進行動作和斷言的快速測試。 |
| 處理複雜的方案 | 自動化複雜方案是具有挑戰性的,容易出現頻繁的失敗。 | 通過關注預期的測試結果來促進複雜方案的測試。 |
| 測試寫作時間 | 僅在功能的完整開發後才能編寫測試。 | 啟用測試驅動的開發(TDD)方法,允許與功能開發或之前或之前的測試寫作。 |
locator.blurlocator.boundingBoxlocator.checklocator.clearlocator.clicklocator.countlocator.filllocator.getAttributelocator.innerHTMLlocator.innerTextlocator.inputValuelocator.isCheckedlocator.isEditablelocator.isEnabledlocator.isVisiblelocator.textContentlocator.uncheckpage.goto添加新操作很容易:只需更新src/completeTask.ts中的functions即可。
這個庫是免費的。但是,使用OpenAi有一些成本。您可以在此處找到有關定價的更多信息:https://openai.com/pricing/。
以https://ray.run/為例,使用GPT-4 Turbo(使用GPT-3.5 Turbo)運行測試步驟的成本約為0.01美元。
低成本部分是因為auto-playwright使用HTML消毒來減少有效載荷大小,例如,下面的有效載荷將提交https://ray.run/。
自然,價格會根據有效載荷而變化很大。
< div class =" cYdhWw dKnOgO geGbZz bGoBgk jkEels " >
< div class =" kSmiQp fPSBzf bnYmbW dXscgu xJzwH jTWvec gzBMzy " >
< h1 class =" fwYeZS fwlORb pdjVK bccLBY fsAQjR fyszFl WNJim fzozfU " >
Learn Playwright
</ h1 >
< h2 class =" cakMWc ptfck bBmAxp hSiiri xJzwS gnfYng jTWvec fzozfU " >
Resources for learning end-to-end testing using Playwright automation
framework
</ h2 >
< div
class =" bLTbYS gvHvKe cHEBuD ddgODW jsxhGC kdTEUJ ilCTXp iQHbtH yuxBn ilIXfy gPeiPq ivcdqp isDTsq jyZWmS ivdkBK cERSkX hdAwi ezvbLT jNrAaV jsxhGJ fzozCb "
> </ div >
</ div >
< div class =" cYdhWw dpjphg cqUdSC fasMpP " >
< a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /blog "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Blog </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Learn in depth subjects about end-to-end testing. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /ask "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Ask AI </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Ask ChatGPT Playwright questions. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /tools "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Dev Tools </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > All-in-one toolbox for QA engineers. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /jobs "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > QA Jobs </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Handpicked QA and Automation opportunities. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /questions "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Questions </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Ask AI answered questions about Playwright. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /discord-forum "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Discord Forum </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Archive of Discord Forum posts about Playwright. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /videos "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Videos </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Tutorials, conference talks, and release videos. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /browser-extension "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > Browser Extension </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > GUI for generating Playwright locators. </ p >
</ div > </ a
> < a
class =" gacSWM dCgFix conipm knkqUc bddCnd dTKJOB leOtqz hEzNkW fNBBKe jTWvec fIMbrO fzozfU group "
href =" /wiki "
> < div class =" plfYl bccLBY hSiiri fNBpvX " > QA Wiki </ div >
< div class =" jqqjPD fWDXZB pKTba bBmAxp hSiiri evbPEu " >
< p > Definitions of common end-to-end testing terms. </ p >
</ div > </ a
>
</ div >
< div
class =" kSmiQp fPSBzf pKTba eTDpsp legDhJ hSiiri hdaZLM jTWvec gzBMzy bGySga fzoybr "
>
< p class =" dXhlDK leOtqz glpWRZ fNCcFz " >
Use < kbd class =" bWhrAL XAzZz cakMWc bUyOMB bmOrOm fyszFl dTmriP " > ⌘ </ kbd > +
< kbd > k </ kbd > + "Tools" to quickly access all tools.
</ p >
</ div >
</ div > auto功能使用Sanitize-HTML在將頁面的HTML消毒之前,然後將其發送到OpenAI。這樣做是為了降低成本並提高生成的文本的質量。
該項目從Zerostep汲取了靈感。 Zerostep提供了類似的API,但通過其專有後端具有更強大的實現。 Auto Playwright的創建是為了探索Zerostep的基礎技術,並為其軟件的開源版本建立基礎。對於生產環境,我建議選擇Zerostep。
這是自動劇作家和Zerostep的並排比較:
| 標準 | 自動劇作家 | Zerostep |
|---|---|---|
| 使用OpenAI API | 是的 | 否1 |
| 使用普通文本提示 | 是的 | 不 |
使用functions SDK | 是的 | 不 |
| 使用HTML消毒 | 是的 | 不 |
| 使用劇作家API | 是的 | 否2 |
| 使用屏幕截圖 | 不 | 是的 |
| 使用隊列 | 不 | 是的 |
| 使用Websockets | 不 | 是的 |
| 快照 | html | dom |
| 實現並行性 | 不 | 是的 |
| 允許滾動 | 不 | 是的 |
| 提供固定裝置 | 不 | 是的 |
| 執照 | 麻省理工學院 | 麻省理工學院 |
MIT License
Copyright (c) 2023 Reflect Software Inc
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
使用Zerostep專有API。 ↩
使用一些劇作家API,但主要依賴於Chrome DevTools協議(CDP)。 ↩