-
Notifications
You must be signed in to change notification settings - Fork 898
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CDSSM model doesnt work #800
Comments
#481 may help with your memory issue. |
I figured out that i can use "with_word_hashing=False" for large datasets, but if i want to use word hashing then it is not possible with large datasets right? Please confirm. |
Word hashing takes a lot of space. Unless you have HUGE memory in your computer, it's not possible to do it as a part of the preprocessing process. What do you mean by "CDSSM doesn't work"? |
did you run the wikiqa/cdssm ipynb? I could not reproduce this error with matchzoo 2.2. did you modify the notebook? I think this error means that the labels are not in the correct form |
This is why you failed. You loaded the ranking dataset, but set the task to classification. |
This is the code that i used for the CDSSM model. import keras print('data loading ...') preprocessor = mz.preprocessors.CDSSMPreprocessor(fixed_length_left=10, fixed_length_right=10) train_processed = preprocessor.fit_transform(train_pack_raw) model = mz.models.CDSSM() pred_x, pred_y = train_processed[:].unpack() history = model.fit_generator(train_generator, epochs=20, callbacks=[evaluate], workers=1, use_multiprocessing=False) print("done") |
Your matchzoo is outdated. Please upgrade it. Matchzoo 2.1 does not work with keras 2.3.1 |
Can i clone the latest version from github? when i do a pip install, its installing an older version of match zoo. Please let me know. |
did you install matchzoo with this command? could you show us the log? |
Describe the Question
I was trying to implement the CDSSM model with the toy dataset provided and I get the following error.
![image](https://user-images.githubusercontent.com/43016846/68265275-96a80e80-0019-11ea-9457-cbf5bb6d3d49.png)
Also the word hashing (preprocessing) is consuming a lot of memory. Is there a hack around to have less memory usage and have word hashing?
I followed the tutorials in the Matchzoo and used the code of CDSSM from wikiqa (https://github.com/NTMC-Community/MatchZoo/tree/master/tutorials/wikiqa)
P.S - I played around with other models like knrm, convknrm, dssm, arcII, duet and mvlstm and they all worked. I only have issue with the CDSSM model. I followed the tutorials for all the other models as well.
The text was updated successfully, but these errors were encountered: