nlprule下載nlprule源代碼下載

nlprule

其他源碼

Release 0.6.4

下載

nlprule

用Rust編寫的快速，低資源的自然語言處理和錯誤校正庫。 NLPrule使用Languagetool的資源來實現NLP規則和查找的方法。

python用法

安裝： pip install nlprule

使用：

 from nlprule import Tokenizer , Rules

tokenizer = Tokenizer . load ( "en" )
rules = Rules . load ( "en" , tokenizer )

 rules . correct ( "He wants that you send him an email." )
# returns: 'He wants you to send him an email.'

rules . correct ( "I can due his homework." )
# returns: 'I can do his homework.'

for s in rules . suggest ( "She was not been here since Monday." ):
    print ( s . start , s . end , s . replacements , s . source , s . message )
# prints:
# 4 16 ['was not', 'has not been'] WAS_BEEN.1 Did you mean was not or has not been?

 for sentence in tokenizer . pipe ( "A brief example is shown." ):
    for token in sentence :
        print (
            repr ( token . text ). ljust ( 10 ),
            repr ( token . span ). ljust ( 10 ),
            repr ( token . tags ). ljust ( 24 ),
            repr ( token . lemmas ). ljust ( 24 ),
            repr ( token . chunks ). ljust ( 24 ),
        )
# prints:
# 'A'        (0, 1)     ['DT']                   ['A', 'a']               ['B-NP-singular']       
# 'brief'    (2, 7)     ['JJ']                   ['brief']                ['I-NP-singular']       
# 'example'  (8, 15)    ['NN:UN']                ['example']              ['E-NP-singular']       
# 'is'       (16, 18)   ['VBZ']                  ['be', 'is']             ['B-VP']                
# 'shown'    (19, 24)   ['VBN']                  ['show', 'shown']        ['I-VP']                
# '.'        (24, 25)   ['.', 'PCT', 'SENT_END'] ['.']                    ['O']

生鏽的用法

推薦設置：

Cargo.toml

[ dependencies ]
nlprule = " <version> "

[ build-dependencies ]
nlprule-build = " <version> " # must be the same as the nlprule version!

build.rs

 fn main ( ) -> Result < ( ) , nlprule_build :: Error > {
    println ! ( "cargo:rerun-if-changed=build.rs" ) ;

    nlprule_build :: BinaryBuilder :: new (
        & [ "en" ] ,
        std :: env :: var ( "OUT_DIR" ) . expect ( "OUT_DIR is set when build.rs is running" ) ,
    )
    . build ( ) ?
    . validate ( )
}

src/main.rs

 use nlprule :: { Rules , Tokenizer , tokenizer_filename , rules_filename } ;

fn main ( ) {
    let mut tokenizer_bytes : & ' static [ u8 ] = include_bytes ! ( concat! (
        env! ( "OUT_DIR" ) ,
        "/" ,
        tokenizer_filename! ( "en" )
    ) ) ;
    let mut rules_bytes : & ' static [ u8 ] = include_bytes ! ( concat! (
        env! ( "OUT_DIR" ) ,
        "/" ,
        rules_filename! ( "en" )
    ) ) ;

    let tokenizer = Tokenizer :: from_reader ( & mut tokenizer_bytes ) . expect ( "tokenizer binary is valid" ) ;
    let rules = Rules :: from_reader ( & mut rules_bytes ) . expect ( "rules binary is valid" ) ;

    assert_eq ! (
        rules . correct ( "She was not been here since Monday." , & tokenizer ) ,
        String :: from ( "She was not here since Monday." )
    ) ;
}

nlprule和nlprule-build版本保持同步。

主要功能

基於規則的語法誤差通過多個規則進行校正。
文本處理管道進行句子細分，言論的一部分標記，lemmatization，塊狀和歧義。
支持英語，德語和西班牙語。
拼寫檢查。（進行中）

目標

一個在下游任務中應用拼寫檢查和語法誤差校正的地方。
快速，低資源的NLP適合運行：
1. 作為更複雜（IE ML）方法的前 /後處理步驟。
2. 在另一個具有低開銷的應用程序的背景下。
3. 通過WebAssembly在瀏覽器中的客戶端。
100％生鏽代碼和依賴關係。

與Languagetool進行比較

	\|歧義規則\|	\|語法規則\|	LT版本	nlprule時間	語言時間
英語	843（100％）	3725（〜85％）	5.2	1	1.7-2.0
德語	486（100％）	2970（〜90％）	5.2	1	2.4-2.8
西班牙語	實驗支持。尚未完全測試。