tensorlayer tricks
1.0.0
儘管深度學習的研究繼續改善世界，但我們使用大量技巧每天實現算法。
這是使用Tensorlayer的技巧的摘要。如果您發現在實踐中特別有用的技巧，請打開拉動請求將其添加到文檔中。如果我們發現它是合理和驗證的，我們將合併它。
tensorlayer您可以通過示為git clone https://github.com/zsdonghao/tensorlayer.git下載整個存儲庫。pip安裝，我們建議您安裝主版本is_fix設置為True在Dropoutlayer中，並通過重複使用參數來構建訓練/測試的不同圖表。您還可以為不同圖形設置不同的batch_size和噪聲概率。當您使用Gaussiannoiselayer，batchnormlayer等。這是一個示例： def mlp ( x , is_train = True , reuse = False ):
with tf . variable_scope ( "MLP" , reuse = reuse ):
net = InputLayer ( x , name = 'in' )
net = DropoutLayer ( net , 0.8 , True , is_train , name = 'drop1' )
net = DenseLayer ( net , n_units = 800 , act = tf . nn . relu , name = 'dense1' )
net = DropoutLayer ( net , 0.8 , True , is_train , name = 'drop2' )
net = DenseLayer ( net , n_units = 800 , act = tf . nn . relu , name = 'dense2' )
net = DropoutLayer ( net , 0.8 , True , is_train , name = 'drop3' )
net = DenseLayer ( net , n_units = 10 , act = tf . identity , name = 'out' )
logits = net . outputs
net . outputs = tf . nn . sigmoid ( net . outputs )
return net , logits
x = tf . placeholder ( tf . float32 , shape = [ None , 784 ], name = 'x' )
y_ = tf . placeholder ( tf . int64 , shape = [ None , ], name = 'y_' )
net_train , logits = mlp ( x , is_train = True , reuse = False )
net_test , _ = mlp ( x , is_train = False , reuse = True )
cost = tl . cost . cross_entropy ( logits , y_ , name = 'cost' )更多在這裡。
train_vars = tl . layers . get_variables_with_name ( 'MLP' , True , True )
train_op = tf . train . AdamOptimizer ( learning_rate = 0.0001 ). minimize ( cost , var_list = train_vars ) layers = tl . layers . get_layers_with_name ( network , "MLP" , True )如果您的數據集很大，則數據加載和數據增加將成為底層並減慢培訓。為了加快數據處理，您可以：
如果您的數據大小足夠小，可以輸入計算機的內存，並且數據增加很簡單。輕鬆調試，您可以：
tl.models x = tf . placeholder ( tf . float32 , [ None , 224 , 224 , 3 ])
# get the whole model
vgg = tl . models . VGG16 ( x )
# restore pre-trained VGG parameters
sess = tf . InteractiveSession ()
vgg . restore_params ( sess )
# use for inferencing
probs = tf . nn . softmax ( vgg . outputs ) x = tf . placeholder ( tf . float32 , [ None , 224 , 224 , 3 ])
# get VGG without the last layer
vgg = tl . models . VGG16 ( x , end_with = 'fc2_relu' )
# add one more layer
net = tl . layers . DenseLayer ( vgg , 100 , name = 'out' )
# initialize all parameters
sess = tf . InteractiveSession ()
tl . layers . initialize_global_variables ( sess )
# restore pre-trained VGG parameters
vgg . restore_params ( sess )
# train your own classifier (only update the last layer)
train_params = tl . layers . get_variables_with_name ( 'out' ) x1 = tf . placeholder ( tf . float32 , [ None , 224 , 224 , 3 ])
x2 = tf . placeholder ( tf . float32 , [ None , 224 , 224 , 3 ])
# get VGG without the last layer
vgg1 = tl . models . VGG16 ( x1 , end_with = 'fc2_relu' )
# reuse the parameters of vgg1 with different input
vgg2 = tl . models . VGG16 ( x2 , end_with = 'fc2_relu' , reuse = True )
# restore pre-trained VGG parameters (as they share parameters, we don’t need to restore vgg2)
sess = tf . InteractiveSession ()
vgg1 . restore_params ( sess ) import tensorflow as tf
import tensorlayer as tl
from keras . layers import *
from tensorlayer . layers import *
def my_fn ( x ):
x = Dropout ( 0.8 )( x )
x = Dense ( 800 , activation = 'relu' )( x )
x = Dropout ( 0.5 )( x )
x = Dense ( 800 , activation = 'relu' )( x )
x = Dropout ( 0.5 )( x )
logits = Dense ( 10 , activation = 'linear' )( x )
return logits
network = InputLayer ( x , name = 'input' )
network = LambdaLayer ( network , my_fn , name = 'keras' )
... > >> captions = [ "one two , three" , "four five five" ] # 2个 句 子
> >> processed_capts = []
> >> for c in captions :
> >> c = tl . nlp . process_sentence ( c , start_word = "<S>" , end_word = "</S>" )
> >> processed_capts . append ( c )
> >> print ( processed_capts )
... [[ '<S>' , 'one' , 'two' , ',' , 'three' , '</S>' ],
... [ '<S>' , 'four' , 'five' , 'five' , '</S>' ]] > >> tl . nlp . create_vocab ( processed_capts , word_counts_output_file = 'vocab.txt' , min_word_count = 1 )
... [ TL ] Creating vocabulary .
... Total words : 8
... Words in vocabulary : 8
... Wrote vocabulary file : vocab . txttl.nlp.create_vocab創建的txt詞彙文件創建詞彙對象 > >> vocab = tl . nlp . Vocabulary ( 'vocab.txt' , start_word = "<S>" , end_word = "</S>" , unk_word = "<UNK>" )
... INFO : tensorflow : Initializing vocabulary from file : vocab . txt
... [ TL ] Vocabulary from vocab . txt : < S > < / S > < UNK >
... vocabulary with 10 words ( includes start_word , end_word , unk_word )
... start_id : 2
... end_id : 3
... unk_id : 9
... pad_id : 0然後，您可以將單詞映射到ID或VICE經文如下：
> >> vocab . id_to_word ( 2 )
... 'one'
> >> vocab . word_to_id ( 'one' )
... 2
>> > vocab . id_to_word ( 100 )
... '<UNK>'
> >> vocab . word_to_id ( 'hahahaha' )
... 9 > >> sequences = [[ 1 , 1 , 1 , 1 , 1 ],[ 2 , 2 , 2 ],[ 3 , 3 ]]
> >> sequences = tl . prepro . pad_sequences ( sequences , maxlen = None ,
... dtype = 'int32' , padding = 'post' , truncating = 'pre' , value = 0. )
... [[ 1 1 1 1 1 ]
... [ 2 2 2 0 0 ]
... [ 3 3 0 0 0 ]]sequence_length > >> data = [[ 1 , 2 , 0 , 0 , 0 ], [ 1 , 2 , 3 , 0 , 0 ], [ 1 , 2 , 6 , 1 , 0 ]]
> >> o = tl . layers . retrieve_seq_length_op2 ( data )
> >> sess = tf . InteractiveSession ()
> >> tl . layers . initialize_global_variables ( sess )
> >> print ( o . eval ())
... [ 2 3 4 ]tl.files.load_and_assign_npz還原tl.files.load_and_assign_npz_dict還原tl.files.load_ckpt還原。 TL可以與其他TF包裝器進行交互，這意味著，如果您找到其他包裝器實施的一些代碼或模型，則可以使用它！
BatchNormLayer的decay默認值為0.9，大型數據集設置為0.999。
