精通
英语
和
开源
,
擅长
开发
与
培训
,
胸怀四海
第一信赖
训练时遇到莫名其妙的错误,这里列举出来一个,我是解决了,如果同行遇到这样的错误,不知道如何解决,请联系锐英源,手机13803810136。以前还经常问kalid邮件列表里的专家,现在自己也能想到办法了。
steps/train_lda_mllt.sh:  Estimating MLLT
steps/diagnostic/analyze_alignments.sh  --cmd run.pl data/lang exp/nnet2_online/tri5b
steps/diagnostic/analyze_alignments.sh:  see stats in exp/nnet2_online/tri5b/log/analyze_alignments.log
4 warnings in  exp/nnet2_online/tri5b/log/mupdate.*.log
12957 warnings in  exp/nnet2_online/tri5b/log/update.*.log
44 warnings in  exp/nnet2_online/tri5b/log/questions.log
140 warnings in  exp/nnet2_online/tri5b/log/acc.*.*.log
56 warnings in  exp/nnet2_online/tri5b/log/init_model.log
1 warnings in  exp/nnet2_online/tri5b/log/compile_questions.log
3 warnings in  exp/nnet2_online/tri5b/log/build_tree.log
374 warnings in  exp/nnet2_online/tri5b/log/align.*.*.log
exp/nnet2_online/tri5b:  nj=10 align prob=-453.81 over 7.94h [retry=1.3%, fail=0.1%] states=61  gauss=10019 tree-impr=6.82 lda-sum=12009.50 mllt:impr,logdet=11479487.01,187.19
steps/train_lda_mllt.sh:  Done training system with LDA+MLLT features in exp/nnet2_online/tri5b
steps/online/nnet2/train_diag_ubm.sh  --cmd run.pl --nj 1 --num-frames 400000 data/train_si84_hires 256  exp/nnet2_online/tri5b exp/nnet2_online/diag_ubm
steps/online/nnet2/train_diag_ubm.sh:  Directory exp/nnet2_online/diag_ubm already exists. Backing up diagonal UBM in  exp/nnet2_online/diag_ubm/backup.Fln
steps/online/nnet2/train_diag_ubm.sh:  initializing model from E-M in memory, 
steps/online/nnet2/train_diag_ubm.sh:  starting from 128 Gaussians, reaching 256;
steps/online/nnet2/train_diag_ubm.sh:  for 20 iterations, using at most 400000 frames of data
Getting  Gaussian-selection info
steps/online/nnet2/train_diag_ubm.sh:  will train for 4 iterations, in parallel over
steps/online/nnet2/train_diag_ubm.sh:  1 machines, parallelized with 'run.pl'
steps/online/nnet2/train_diag_ubm.sh:  Training pass 0
steps/online/nnet2/train_diag_ubm.sh:  Training pass 1
steps/online/nnet2/train_diag_ubm.sh:  Training pass 2
steps/online/nnet2/train_diag_ubm.sh:  Training pass 3
steps/online/nnet2/train_ivector_extractor.sh  --cmd run.pl --nj 1 data/train_si284_hires exp/nnet2_online/diag_ubm  exp/nnet2_online/extractor
steps/online/nnet2/train_ivector_extractor.sh:  Directory exp/nnet2_online/extractor already exists. Backing up iVector  extractor in exp/nnet2_online/extractor/backup.9x1
steps/online/nnet2/train_ivector_extractor.sh:  doing Gaussian selection and posterior computation
run.pl: 4 / 4  failed, log is in exp/nnet2_online/extractor/log/post.*.log
(base)  server@server-PowerEdge-R740:~/kaldi-trunk/egs/wsj/s5/exp/nnet2_online/extractor/log$  cat post.1.log 
#  gmm-global-get-post --n=5 --min-post=0.025  exp/nnet2_online/extractor/final.dubm "ark,s,cs:apply-cmvn-online  --config=exp/nnet2_online/extractor/online_cmvn.conf  --spk2utt=ark:data/train_si284_hires/split4/1/spk2utt  exp/nnet2_online/extractor/global_cmvn.stats  scp:data/train_si284_hires/split4/1/feats.scp ark:- | splice-feats  --left-context=3 --right-context=3 ark:- ark:- | transform-feats exp/nnet2_online/extractor/final.mat  ark:- ark:- | subsample-feats --n=2 ark:- ark:- |" ark:- | scale-post  ark:- 0.2 "ark:|gzip -c >exp/nnet2_online/extractor/post.1.gz" 
# Started at Wed  Jun 30 11:15:03 CST 2021
#
scale-post ark:-  0.2 'ark:|gzip -c >exp/nnet2_online/extractor/post.1.gz' 
gmm-global-get-post  --n=5 --min-post=0.025 exp/nnet2_online/extractor/final.dubm  'ark,s,cs:apply-cmvn-online  --config=exp/nnet2_online/extractor/online_cmvn.conf  --spk2utt=ark:data/train_si284_hires/split4/1/spk2utt exp/nnet2_online/extractor/global_cmvn.stats  scp:data/train_si284_hires/split4/1/feats.scp ark:- | splice-feats  --left-context=3 --right-context=3 ark:- ark:- | transform-feats  exp/nnet2_online/extractor/final.mat ark:- ark:- | subsample-feats --n=2 ark:-  ark:- |' ark:- 
LOG  (gmm-global-get-post[5.5.839~8-0c6a]:ComputeGconsts():diag-gmm.cc:119) num  gs256dim40offset-36.7575
transform-feats  exp/nnet2_online/extractor/final.mat ark:- ark:- 
subsample-feats  --n=2 ark:- ark:- 
splice-feats  --left-context=3 --right-context=3 ark:- ark:- 
apply-cmvn-online  --config=exp/nnet2_online/extractor/online_cmvn.conf  --spk2utt=ark:data/train_si284_hires/split4/1/spk2utt  exp/nnet2_online/extractor/global_cmvn.stats  scp:data/train_si284_hires/split4/1/feats.scp ark:- 
ASSERTION_FAILED  (apply-cmvn-online[5.5.839~8-0c6a]:SmoothOnlineCmvnStats():online-feature.cc:410)  Assertion failed: (global_count > 0.0)
[ Stack-Trace: ]
  /home/server/kaldi-trunk/src/lib/libkaldi-base.so(kaldi::MessageLogger::LogMessage()  const+0xb42) [0x7f901d3c0692]
  /home/server/kaldi-trunk/src/lib/libkaldi-base.so(kaldi::KaldiAssertFailure_(char  const*, char const*, int, char const*)+0x6e) [0x7f901d3c138e]
  /home/server/kaldi-trunk/src/lib/libkaldi-feat.so(kaldi::OnlineCmvn::SmoothOnlineCmvnStats(kaldi::MatrixBase<double>  const&, kaldi::MatrixBase<double> const&,  kaldi::OnlineCmvnOptions const&, kaldi::MatrixBase<double>*)+0x21b)  [0x7f901dacbeeb]
  /home/server/kaldi-trunk/src/lib/libkaldi-feat.so(kaldi::OnlineCmvn::GetFrame(int,  kaldi::VectorBase<float>*)+0x12a) [0x7f901dacd944]
  apply-cmvn-online(main+0x6a2)  [0x55b26bf57c5b]
  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)  [0x7f901c82abf7]
  apply-cmvn-online(_start+0x2a)  [0x55b26bf574aa]
Aborted (core  dumped)
  LOG  (transform-feats[5.5.839~8-0c6a]:main():transform-feats.cc:161) Applied  transform to 0 utterances; 0 had errors.
  LOG  (subsample-feats[5.5.839~8-0c6a]:main():subsample-feats.cc:115) Processed 0  feature matrices; 0 with errors.
  LOG  (subsample-feats[5.5.839~8-0c6a]:main():subsample-feats.cc:117) Processed 0 input  frames and 0 output frames.
  LOG  (gmm-global-get-post[5.5.839~8-0c6a]:main():gmm-global-get-post.cc:115) Done 0  files, 0 with errors, average UBM log-likelihood is -nan over 0 frames.
  WARNING  (gmm-global-get-post[5.5.839~8-0c6a]:Close():kaldi-io.cc:515) Pipe  apply-cmvn-online --config=exp/nnet2_online/extractor/online_cmvn.conf  --spk2utt=ark:data/train_si284_hires/split4/1/spk2utt  exp/nnet2_online/extractor/global_cmvn.stats  scp:data/train_si284_hires/split4/1/feats.scp ark:- | splice-feats --left-context=3  --right-context=3 ark:- ark:- | transform-feats  exp/nnet2_online/extractor/final.mat ark:- ark:- | subsample-feats --n=2 ark:-  ark:- | had nonzero return status 256
  ERROR  (gmm-global-get-post[5.5.839~8-0c6a]:~SequentialTableReaderArchiveImpl():util/kaldi-table-inl.h:678)  TableReader: error detected closing archive 'apply-cmvn-online  --config=exp/nnet2_online/extractor/online_cmvn.conf  --spk2utt=ark:data/train_si284_hires/split4/1/spk2utt  exp/nnet2_online/extractor/global_cmvn.stats scp:data/train_si284_hires/split4/1/feats.scp  ark:- | splice-feats --left-context=3 --right-context=3 ark:- ark:- |  transform-feats exp/nnet2_online/extractor/final.mat ark:- ark:- |  subsample-feats --n=2 ark:- ark:- |'
[ Stack-Trace: ]
  /home/server/kaldi-trunk/src/lib/libkaldi-base.so(kaldi::MessageLogger::LogMessage()  const+0xb42) [0x7fe1ae1d4692]
  gmm-global-get-post(kaldi::MessageLogger::LogAndThrow::operator=(kaldi::MessageLogger  const&)+0x21) [0x55660d95d4e1]
  gmm-global-get-post(kaldi::SequentialTableReaderArchiveImpl<kaldi::KaldiObjectHolder<kaldi::Matrix<float>  > >::~SequentialTableReaderArchiveImpl()+0x11c) [0x55660d960fa4]
  gmm-global-get-post(kaldi::SequentialTableReaderArchiveImpl<kaldi::KaldiObjectHolder<kaldi::Matrix<float>  > >::~SequentialTableReaderArchiveImpl()+0x9) [0x55660d9612b9]
  gmm-global-get-post(kaldi::SequentialTableReader<kaldi::KaldiObjectHolder<kaldi::Matrix<float>  > >::~SequentialTableReader()+0x12) [0x55660d9621cc]
  gmm-global-get-post(main+0xd52)  [0x55660d95c71c]
  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)  [0x7fe1ad63ebf7]
  gmm-global-get-post(_start+0x2a)  [0x55660d95b8ea]
terminate called  after throwing an instance of 'kaldi::KaldiFatalError'
  what():   kaldi::KaldiFatalError
  LOG  (scale-post[5.5.839~8-0c6a]:main():scale-post.cc:79) Done 0 posteriors;  0 had no scales.
  # Accounting:  time=1 threads=1
  # Ended (code 1)  at Wed Jun 30 11:15:04 CST 2021, elapsed time 1 seconds